6 Steps Data Analytics process to Transform Product Development
Step 1: Ask the Right Questions
A new project management software tool aimed at small businesses. While the tool had a strong user base, customer reviews showed that many users were unhappy with the user interface (UI), finding it difficult to navigate and frustrating to use.
the data analyst:
Takes the time to fully understand stakeholder expectations
Defines the problem to be solved
Decides which questions to answer in order to solve the problem
Qualifying stakeholder expectations means determining who the stakeholders are, what they want, when they want it, why they want it, and how best to communicate with them. Defining the problem means looking at the current state and identifying the ways in which it’s different from the ideal state. With expectations qualified and the problem defined, you can derive questions that will help achieve these goals.
They began by asking:
- What specific features do users find difficult to navigate?
- Are there any recurring themes in user feedback or support tickets?
- Which features do users engage with the most, and which do they ignore?
- How can we redesign the UI to address these pain points?
Step 2: Prepare for Data Collection
Data Analyst decide what data need to be collected in order to answer all questions and how to organize those data so that it is useful.
- What metrics to measure
- Locate data in your database
- Create security measures to protect that data
Here’s what the preparation looked like:
- Developed a set of questions asking users to rate their experience with the software’s UI.
- Gathered historical support tickets that detailed common issues or feature requests.
- Set up heatmap software to track user interactions on the software interface.
- Established clear guidelines for how to share and manage the collected data, ensuring privacy and confidentiality.
Step 3: Process the Data
This usually means:
Cleaning data
Transforming data into a more useful format
Combining two or more datasets to make information more complete
Removing outliers (data points that could skew the information)
After data analysts process data, they check the data they prepared to make sure it’s complete and correct.
The analysts first ensured that:
- User responses from the survey were complete and free of any inconsistencies.
- Support tickets were categorized by type of issue (e.g., UI issues, performance issues, feature requests).
- Heatmap data was properly aggregated to ensure that it showed clear, actionable insights (e.g., which buttons or features users clicked the most and where they abandoned the app).
The team took steps to clean and anonymize the data to protect user privacy and prevent any personal information from being disclosed.This phase is all about getting the details right. Accordingly, the data analyst will refine strategies for verifying and sharing their data cleaning with stakeholders.
Step 4: Analyze the Data
Here’s what they found:
- Confusing Navigation: A significant users struggled with navigating through different features, particularly the settings menu and project view.
- Low Engagement with Key Features: Heatmap analysis showed that users were not interacting with key features, such as task prioritization tools and the calendar view, likely due to poor visibility and unclear labeling.
- Frustration with Onboarding: Many users unsure of how to get started and use the software effectively.
- Performance Complaints: While not directly related to the UI, many users also mentioned performance issues like slow load times.
Step 5: Share the Findings
Here’s how they shared the results:
- The analysts presented the findings to the product, design, and engineering teams in a well-organized report, highlighting the main pain points and suggested improvements.
- They visualized the findings using graphs and heatmaps to make it easy for stakeholders to see where users were struggling most.
- The team then held a meeting with the design team to discuss possible UI changes and how to prioritize the most impactful improvements.
- They also made sure to present the data in a way that was understandable to non-technical stakeholders, ensuring alignment across teams.
Step 6: Act on the Insights
Armed with clear insights, the product and design teams began working on a plan to improve the user experience.
Here’s what they decided to do:
- Simplify the Navigation: The design team redesigned the software’s navigation to make it more intuitive, reducing the number of steps needed to access key features and ensuring that frequently-used tools were more accessible.
- Highlight Key Features: They revamped the UI to make important features, like task prioritization and the calendar view, more prominent and easier to find. Labels were clarified, and buttons were made more visible.
- Improve Onboarding: They created a more comprehensive onboarding process, including tooltips, walkthroughs, and a clear, simple tutorial to guide new users through the software.
- Address Performance Issues: The engineering team worked to optimize the software’s performance by improving load times and addressing minor bugs that were affecting the user experience.
The changes were rolled out over a series of software updates, with the team continuously gathering user feedback after each update.
Results: A Smarter, More User-Friendly Product
A few months later, the product team sent out another survey and reviewed the heatmap data to see how the changes had impacted user experience. The results were clear: user satisfaction had increased by 25%. The number of support tickets related to UI issues decreased significantly, and the company saw higher user engagement with previously underused features.
What Is a Data Ecosystem?
At its core, an ecosystem is a network of elements that interact with one another to produce an outcome. Say natural environments like rainforests or oceans, where plants, animals, and weather patterns all work together. Similarly data lives in its own ecosystem, where different elements—hardware, software, and people—interact to generate, store, process, and analyze data. This interconnected system is what makes modern data analytics possible.
The Building Blocks of the Data Ecosystem
Hardware and Software: Hardware store data, while software refers to the applications that help you manipulate and visualize data (such as databases, data visualization tools, and analytics platforms).
The Cloud: This allows data to be stored, shared, and accessed remotely, offering greater flexibility and scalability for businesses and analysts.
People: An analyst or someone in a different role, interact with this data to draw conclusions, solve problems, and guide decisions. As a data analyst, your job is to navigate this ecosystem and extract valuable insights to help businesses thrive.
Real-World Applications of Data Ecosystems
Retail: The store’s database contains a wealth of information: customer names, addresses, purchase history, and reviews. By analyzing this data, you can predict which products customers are likely to buy next, ensuring the store is stocked accordingly. A well-managed data ecosystem enables the business to stay ahead of trends and meet customer demands before they arise.
Agriculture: Farmers collect data about weather patterns, soil conditions, and crop health, which can help predict crop yields. By analyzing this data, agricultural companies can help farmers make more informed decisions, leading to better crop management and improved sustainability practices.
Misconceptions in the Data World
Data Scientist vs. Data Analyst
Data Scientists are like explorers—they create new ways to model and understand raw data. They design algorithms, build predictive models, and often work with machine learning techniques to make sense of complex data.
Data Analysts, on the other hand, focus on finding answers to specific business questions by analyzing existing data. If data science is about asking new questions, data analysis is about answering existing ones.
Data Analysis vs. Data Analytics
Data Analysis refers to the process of collecting, cleaning, transforming, and organizing data to make sense of it. It’s about drawing conclusions and making predictions from data to guide decisions. For example, as a data analyst, you might analyze sales data to understand trends and inform inventory decisions.
Data Analytics, however, is a broader term that encompasses all activities related to data. This includes not only the analysis but also the tools, technologies, and methods that enable the entire process of managing and utilizing data. Data analytics can include techniques like data mining, predictive analytics, and even machine learning, making it a more comprehensive field.
5 Essential Analytical Skills You Already Have (And How They Help in Data Analysis)
1. Curiosity: The Desire to Learn and Explore
Curiosity is about asking questions and seeking answers. Curious people tend to explore new challenges, dive deep into subjects, and constantly strive to expand their knowledge.
Example: Say you decide to learn how to bake bread. The first thing you do is research recipes, watch tutorials, and ask questions like, “What ingredients are essential?” or “How does the yeast work?” As a data analyst, that same curiosity fuels your exploration of data to uncover insights and patterns.
2. Understanding Context: Seeing the Bigger Picture
Context is key in understanding how data or information fits into a larger framework. It’s about knowing not just the facts, but the situation surrounding them. A simple example is the idea of counting numbers. If I say “1, 2, 3, 4, 5,” you understand the sequence because it makes sense in the context of counting. But if I say “1, 2, 4, 5, 3,” you can quickly identify that the “3” is out of place.
Example: A sharp drop in sales might not just be a result of poor performance—it could also be due to external factors like a competitor’s promotion or a market downturn. Understanding this context is crucial for proper data analysis.
3. A Technical Mindset: Breaking Down Complex Tasks
A technical mindset is all about breaking complex problems into smaller, manageable pieces. It involves approaching tasks in a systematic, step-by-step way.
Example: Think about paying your bills. You don’t just throw all your bills in a pile and pay them randomly. Instead, you organize them by due dates, then check your bank balance to see if you can pay everything off or need to wait until your next paycheck. Each of these steps is part of a process, making the task more manageable and ensuring you stay on top of things.
4. Data Design: Organizing Information Effectively
Data design is the skill of organizing information in a clear, logical way that allows you to find and use it easily. Whether it’s arranging data in a database or organizing information in your daily life, how you structure things makes a huge difference.
Example: Take a look at how you organize the contacts on your phone. This design helps you quickly find a contact when you need it, saving you time. In data analysis, this skill translates into organizing data in a way that makes it easy to analyze and draw insights from. Whether you’re creating a database or designing a report, clear and logical organization is key to making your work efficient and effective.
5. Data Strategy: Managing People, Processes, and Tools
Data strategy is about effectively managing the people, processes, and tools involved in data analysis. This includes ensuring the right people know how to use the right data, creating a clear process for analysis, and utilizing the right tools for the job.
Example: Think about mowing a lawn. To do it effectively, you follow a strategy:
- People: You, the person mowing the lawn, need to know how to operate the mower safely (perhaps by reading the owner’s manual).
- Process: You prepare the lawn by removing obstacles like sticks or rocks. This clears the way for a smooth mowing process.
- Tools: You check that the lawnmower has enough gas and is in working condition.
In data analysis, a similar strategy is used: managing the team’s knowledge, the steps they take to analyze data, and the tools they use to interpret and present that data.
Unlocking the Power of Analytical Thinking in Data Analysis
Five key aspects of analytical thinking that can help you approach data problems with clarity and precision.
1. Visualization: Turning Data into a Story
Visualization is the graphical representation of information, such as charts, graphs, or maps, to help make complex data easier to understand.
Example:
For example, a data analyst might use a bar chart to display sales growth over time, helping stakeholders easily see which months performed best and which need improvement. In essence, visualization brings data to life and makes insights more accessible.
2. Strategy: Keeping Your Eyes on the Goal
With an abundance of data at our disposal, it’s easy to get lost in the details. A strategic mindset helps data analysts stay focused on the end goal—whether it’s solving a specific problem, finding trends, or informing decision-making.
Example: If a company wants to reduce its customer churn rate, the data analyst will strategically focus on metrics like customer retention, satisfaction scores, and usage patterns to understand what’s contributing to the churn and how to address it.
3. Problem-Orientation: Solving Problems Step by Step
Keeping the problem at the forefront of your mind throughout the entire analysis process. They define the problem clearly, gather relevant data, and follow a systematic process to find the solution.
Example: Let’s say a retail company is losing customers. The analyst will start by asking the right questions, gathering the necessary data, and then narrowing down the cause of the problem.
4. Correlation: Identifying Relationships Between Data Points
Correlation is about recognizing patterns or connections that might not be immediately obvious. But, as a data analyst, it’s crucial to remember that correlation does not imply causation.
Example: An analyst might find a correlation between high temperatures and increased sales of ice cream. But does that mean hot weather causes ice cream sales to increase? Not necessarily. There might be other factors, such as a local festival or promotions, that influence the sales.
5. Big-Picture and Detail-Oriented Thinking: Balancing the Forest and the Trees
As a data analyst, you need to be able to zoom in on the details and zoom out to see the larger context. Big-picture thinking helps you understand the overall goals and objectives, while detail-oriented thinking ensures you can break down the data to solve specific problems.
Example: If you’re analyzing sales data, understanding the company’s strategic goals will help you know what trends to focus on. Detail-oriented thinking helps you analyze specific numbers—like monthly sales figures or customer feedback—which will reveal the underlying story.
Why Thinking in Different Ways is Important
You might naturally excel at one type of thinking, but the best data analysts are versatile thinkers who can adapt to different situations. For example, you might be strong in critical thinking (analyzing the “what” and “why”), but you can also nurture your creative thinking (finding new and innovative solutions) and analytical thinking (systematically solving problems with data).
Analytical thinking allows you to break down a problem step-by-step, while creative thinking helps you come up with new approaches. Critical thinking helps you question assumptions and dive deeper into the root cause of problems.
Analytical thinking: The process of identifying and defining a problem, then solving it by using data in an organized, step-by-step manner
The Five Whys is a great tool for getting to the root cause of a problem. By asking “Why?” five times, you can peel back the layers of a problem and uncover insights that might not be immediately apparent.
Example: Let’s say your company is experiencing declining product sales. Here’s how the Five Whys might help you uncover the root cause:
- Why are sales down? – Customers aren’t purchasing the product.
- Why aren’t customers purchasing? – They complain that the product isn’t available in stores.
- Why isn’t it available? – Supply chain issues are delaying shipments.
- Why are there supply chain issues? – There’s a shortage of key raw materials.
- Why is there a shortage of raw materials? – The supplier’s factory was hit by a natural disaster.
By asking “Why?” five times, you’ve discovered the root cause— a natural disaster affecting the supplier’s ability to deliver. This insight allows you to take proactive steps, like finding alternative suppliers or adjusting marketing strategies until the issue is resolved.
Understanding the Data Life Cycle: From Planning to Destruction
We’ll explore the six key phases of the data life cycle: Plan, Capture, Manage, Analyze, Archive, and Destroy.
1. Planning: Setting the Foundation
Planning happens before any data is even collected and involves deciding on the types of data needed, how that data will be handled, and who will be responsible for it. Essentially, this phase sets the blueprint for the entire project.
Example: Let’s say an electricity provider wants to explore ways to help its customers save energy. In the planning phase, the company would identify the types of data it needs, such as customer electricity usage patterns, building types, and the types of devices being used within those buildings. The company would also decide which team members are responsible for collecting, storing, and analyzing this data. This clear roadmap helps ensure that everyone is aligned on the goals and processes moving forward.
2. Capture: Collecting the Data
Data can come from a variety of sources. Whether from external resources like public datasets or internal company databases, the capture phase is all about gathering the raw data that will later be analyzed.
Example: In our electricity provider example,they might pull historical electricity usage data from their own systems or use external resources like public energy consumption statistics or weather data from the National Climatic Data Center. This data could be stored in databases for easy access and further analysis.
At this stage, ensuring data integrity, credibility, and privacy is crucial.
3. Manage: Maintaining and Securing the Data
Managing data includes the implementation of tools and systems that ensure data is safe, organized, and ready for analysis.
Example: Storing the electricity usage data in a secure database, backed up regularly to prevent loss. The company would also need to ensure that the data is accessible to authorized team members, and implement encryption to protect sensitive customer information.
Data cleansing—removing errors, inconsistencies, or irrelevant data—is an important part of this phase.
4. Analyze: Turning Data into Insights
Data analysts dive in to extract meaningful insights. This is the core function of a data analyst: to use data to solve problems, make decisions, and support business goals.
Example: By analyzing the captured data—looking at electricity usage patterns, types of buildings, and devices used—the data analyst can uncover trends and insights that suggest which customers or building types use the most energy. These insights can inform strategies to encourage energy-saving behaviors among customers, or even identify products or services to help reduce consumption.
In this phase, data visualization plays a key role in helping analysts communicate their findings. Graphs, charts, and other visuals make it easier for stakeholders to understand complex data.
5. Archive: Storing Data for the Future
This is an essential part of the data life cycle, as it allows organizations to keep historical records while ensuring that their active data remains clean and manageable.
Example: Data still has value for future reference—whether for tracking long-term trends or for compliance purposes. So, the company would archive the data in a secure location, ensuring it remains available for future use but doesn’t clutter the active data systems.
6. Destroy: Safeguarding Privacy
It’s an important part of the data life cycle, particularly when it comes to protecting sensitive or outdated information. Destruction is necessary to ensure that data does not fall into the wrong hands or cause privacy issues.
Example: After a set period, the company may no longer need to retain specific customer data. To protect customer privacy, the company would securely destroy the data using data erasure software or by shredding paper files. This ensures that sensitive information cannot be recovered or misused.
Consider fairness
Consider fairness
Following are some strategies that support fair analysis:
Best practice | Explanation | Example |
|---|---|---|
Consider all of the available data | Part of your job as a data analyst is to determine what data is going to be useful for your analysis. Often there will be data that isn’t relevant to what you’re focusing on or doesn’t seem to align with your expectations. But you can’t just ignore it; it’s critical to consider all of the available data so that your analysis reflects the truth and not just your own expectations. | A state’s Department of Transportation is interested in measuring traffic patterns on holidays. At first, they only include metrics related to traffic volumes and the fact that the days are holidays. But the data team realizes they failed to consider how weather on these holidays might also affect traffic volumes. Considering this additional data helps them gain more complete insights. |
Identify surrounding factors | As you’ll learn throughout these courses, context is key for you and your stakeholders to understand the final conclusions of any analysis. Similar to considering all of the data, you also must understand surrounding factors that could influence the insights you’re gaining. | A human resources department wants to better plan for employee vacation time in order to anticipate staffing needs. HR uses a list of national bank holidays as a key part of the data-gathering process. But they fail to consider important holidays that aren’t on the bank calendar, which introduces bias against employees who celebrate them. It also gives HR less useful results because bank holidays may not necessarily apply to their actual employee population. |
Include self-reported data | Self-reporting is a data collection technique where participants provide information about themselves. Self-reported data can be a great way to introduce fairness in your data collection process. People bring conscious and unconscious bias to their observations about the world, including about other people. Using self-reporting methods to collect data can help avoid these observer biases. Additionally, separating self-reported data from other data you collect provides important context to your conclusions! | A data analyst is working on a project for a brick-and-mortar retailer. Their goal is to learn more about their customer base. This data analyst knows they need to consider fairness when they collect data; they decide to create a survey so that customers can self-report information about themselves. By doing that, they avoid bias that might be introduced with other demographic data collection methods. For example, if they had sales associates report their observations about customers, they might introduce any unconscious bias the employees had to the data. |
Use oversampling effectively | When collecting data about a population, it’s important to be aware of the actual makeup of that population. Sometimes, oversampling can help you represent groups in that population that otherwise wouldn’t be represented fairly. Oversampling is the process of increasing the sample size of nondominant groups in a population. This can help you better represent them and address imbalanced datasets. | A fitness company is releasing new digital content for users of their equipment. They are interested in designing content that appeals to different users, knowing that different people may interact with their equipment in different ways. For example, part of their user-base is age 70 or older. In order to represent these users, they oversample them in their data. That way, decisions they make about their fitness content will be more inclusive. |
Think about fairness from beginning to end | To ensure that your analysis and final conclusions are fair, be sure to consider fairness from the earliest stages of a project to when you act on the data insights. This means that data collection, cleaning, processing, and analysis are all performed with fairness in mind. | A data team kicks off a project by including fairness measures in their data-collection process. These measures include oversampling their population and using self-reported data. However, they fail to inform stakeholders about these measures during the presentation. As a result, stakeholders leave with skewed understandings of the data. Learning from this experience, they add key information about fairness considerations to future stakeholder presentations. |
