Consulting, Data Visualization, Microsoft Technologies, Power BI

Stress Cases and Data Visualization

Times are stressful right now. There is an ongoing pandemic affecting people’s health and livelihoods. Schedules are messed up, kids are home. People who aren’t used to working remotely are fumbling through learning how to work from home. And then there are the normal stresses that aren’t taking a break just because there is a pandemic: arguments with family members, home repairs, student loan debt.

Woman with her head in her hands
Life is stressful right now. It’s ok to not be ok.

I recently read the book Design for Real Life by Eric Meyer and Sara Wachter-Boettcher. I discovered it because I read another book by Sara Wachter-Boettcher, Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech that encourages us to look at the technology around us and consider how it can and has negatively affected people. If you work in analytics or web design, I recommend both books. She approaches design more from the context of user interfaces and web apps, but there is a lot of applicability to data visualization.

What’s a Stress Case?

Design for Real Life encourages us to consider stress cases. The idea behind stress cases is to consider our users as full, complex, sometimes distracted, vulnerable, and stressed-out people. They aren’t just happy personas who always click the right buttons and share all of our knowledge and assumptions.

Certain use cases, often labeled as edge cases, often go unidentified or dismissed as too infrequent. User interface designers, including data visualization designers, need to train ourselves to seek out those stress cases and understand how our design choices affect people in those usage scenarios. This helps us to minimize harm done by our design. But also, as stated in the book,

When we make things for people at their worst, they’ll work that much better when people are at their best.

Design for Real Life

There is a toxic trend in tech that we developers make things for ourselves, valuing the developers and designers over the users. Designing for stress cases helps us change that trend. We need to value our users’ time, understand our biases as designers, and make features that match our users’ priorities. A very applicable part of this for data visualization is to consider all the contexts that usage scenarios might happen.

Stress Cases in Data Visualization

Whether you are designing a visualization to be embedded in an app, included in an online news article, or published in a corporate dashboard, you can design for stress cases.

Let’s think about a corporate HR report built in Power BI that shows a prediction of employee retention. HR and team managers may use this information to help them assign projects or give promotions and raises. This dashboard and the conversation around it may address information related to identity (race, gender, sexual orientation). The use of the dashboard may affect someone’s compensation and job satisfaction.

How can we design for stress cases here?

We should be asking if we have the right (appropriate and validated) data to accomplish our goals, not just using whatever we can get our hands on. We definitely need to consider laws against discrimination that might be applicable to how we use demographic data. We need to consider the cost or harm done if we allow users to see these predictions down to the individual employee rather than an aggregation. And we should always be asking what actions people are taking as a result of using our dashboard. Have we considered any unintended consequences?

We want to use plain, easy to understand language. We want to explain our data sources and (at least high-level) how the predictive algorithm works. And we need to explain how we intend for this dashboard to be used. Basically, we want to be as transparent and easy to understand as possible. This dashboard can affect people’s livelihoods and happiness as well as the operational and financial health of the business. These data points are more than just pretty dots – they are people.

We also want to make sure our dashboard is easy to use. Think about digital affordances. We want it to be clear what happens when someone interacts with it. We can get fancy with bookmarks and editing interactions in a Power BI report. But does our intended user understand what is shown when they click a button that loads a bookmark? Can the intended user easily get what they need? Or do they have to jump through hoops to get to the right page and set the right filters?

Let’s say Sarah is a new manager at an organization that is trying to improve a high attrition rate, and she needs this information for a meeting with her boss and peers. She’s sitting in an online meeting using a 3-year-old tablet with 10 minutes of remaining battery life. Her two-year-old child is running around in the next the room. The group is discussing whether they should let some people go and then try to rebuild their teams. Can Sarah easily pull up the dashboard on her tablet and set the correct filters to see attrition numbers and retention factors for her team while trying to put on a brave face in front of her colleagues as she races against the tablet’s remaining battery time?

Visualizing the Coronavirus

Currently, it seems everyone in the data community wants to visualize the spread of COVID-19. Just look at the Power BI Data Stories Gallery. I get the appeal of using new and relevant data for practice, but consider your audience and the consequences of making it public. Do you trust and understand your data? Have you considered how the chart types and colors you choose can misrepresent the data? Who is it helping for you to publish your visualization out into the world? What message is your visualization sending? Is it unintentionally adding stress for your already stressed-out twitter followers? Most of us are not working with city officials or healthcare practitioners. We are visualizing this “for fun”. People are showing off Power BI/Tableau/D3 skills and not necessarily communicating clearly with purpose.

Your twitter friend/follower Frank is really concerned about the pandemic. His daughter lost her job at a restaurant. He’s worried about lining up projects for the next few months. And his wife is immunocompromised and in a higher risk group for getting COVID-19. He feels scared and helpless.

Your friend Dave is worried about his mother who lives alone a few hours away from him. He is bombarded by messages every day that say older people are more at risk of getting COVID-19. His mom says she is fine, but he wants her to come and stay with him. It’s all he’s thought about all day.

If you know nothing about epidemiology and don’t understand the response from various national, state, and local governments in your data, and you publish your viz with a choropleth (filled) map covered in shades of red, how will that affect Frank and Dave and everyone else who reads it? Did you add value to the COVID-19 conversation, or just increase confusion and fear?

Amanda Makulec published Ten Considerations Before You Create Another Chart About COVID-19 with some good advice, including the following.

To sum it up — #vizresponsibly; which may mean not publishing your visualizations in the public domain at all.

Amanda Makulec

Stress cases can be related to a crisis, mundane technology failures, or just situations that are stressful in the context of the user’s life. If you are visualizing data, remember there are people who may be interacting with our visuals in less than ideal circumstances. We need to design for them as much as for our ideal use cases.

If you have real-life examples of designing for stress cases in data visualization, please share them in the comments or tweet me at @mmarie.

Consulting

So You Want to Be A Business Intelligence Consultant

I’ve had a few people ask me recently for advice on getting into business intelligence/analytics consulting. I have been in consulting for 6+ years and have worked for 3 different consulting firms, so I’m sharing my thoughts and experiences here in case they are helpful to someone else. My first response to people asking this question is usually to clarify what they mean by consulting.

What do you envision when you think about consulting?

Female consultant standing with laptop

Do you want to be an independent consultant, work through a placement agency, or work as a full-time employee (FTE) for a consulting firm?

Life looks a little different based upon your answer to this question. I have chosen to work for a consulting firm for a few reasons:

  • I like the stability of a salary and paid time off (vacation, holidays, sick days)
  • I don’t have to do my own accounting/taxes/collections
  • I’m not solely dependent on my sales efforts to bring in work
  • I like having smart coworkers that are in the trenches with me

If you are independent, you are responsible for everything. You must take care of (do yourself or outsource) your own accounting, legal, taxes, collections, sales, etc. Even if you outsource some of these functions, you need to stay somewhat involved – you can’t just throw your financial documents over the fence and wipe your hands of them.

And if you aren’t doing billable work and getting the client to pay you, you aren’t getting paid. This means you need some financial reserves to cover slow months.

My independent consultant friends tell me that sales takes up much more time than they had initially anticipated. They are always working to line up that next engagement to avoid having large gaps between projects. It takes work to build up a sales pipeline. Sometimes you can mitigate this by teaming up with other independent consultants or companies to have a lead referral agreement. They can refer clients to you when they need help in your area of expertise. But there is usually no guarantee of work from these other parties, and sometimes they take a cut of the revenue for sending the work to you. You can outsource a lot of things, but sales is rather difficult because you need the salesperson to understand what you do, the value it provides to clients, and what your values and preferences are.

Another issue you face as an independent consultant (and sometimes as a consultant for a very small consulting company) is how to get interesting projects that require multiple people. Let’s say you have found a potential client that wants to do some really cool analytics, but the project requires three people to work together. You’ll need to have relationships with other consultants with whom you can team up to work the project. And even if you have good people you can work with in general, the work needs to fit into their schedules .

As an FTE, I am still responsible for some sales efforts, but we also have a salesperson and other employees to help. And if I don’t sell anything, I still get paid. This is a great thing for my mortgage and my stress levels.

But as an independent consultant, you also are your own boss. You get to make all of the decisions about how and where to spend your time, what office and tech supplies to purchase, what your website looks like, how much vacation you can take, and who you want to do work for. Want to take a month off to travel the world or spend time with your family? You don’t need anyone else’s permission to do that. But you won’t be getting paid if you aren’t working. One way to mitigate this is to create content that can be sold so you have some passive income in addition to your normal consulting engagements. An example of this is creating a business intelligence course and selling it on your website or through an online learning/education company. You could continue to get revenue/royalties as people purchase the course. But do your homework to get realistic expectations about how much extra income this would mean relative to the time it takes to create and update the content.

If you want to be a contractor where you do full-time, often 3- to 12-month, engagements with a company, you can get your own work or you can go through placement firms. Some placement firms offer benefits and some guarantee of work in exchange for a share of the income resulting from your placement. Others take a flat fee (either from you or from the hiring company). This type of consulting is usually paid based upon an hourly rate.

Do you want to be a consultant who is more project-based where you have a specified scope and budget, and you may work for multiple clients at one time? Or do you want to be embedded full-time in an organization?

All consultants/contractors must complete (most of) the tasks and projects they take on, but managing scope and budget are special skills. Some people enjoy that challenge while others do their best to avoid it. If you are essentially a contractor who spends your time with a single client and does whatever comes up, you may have to spend less time and effort managing scope and budget. If your work is more project-based, this becomes an important part of your work.

If you do project-based consulting, you may occasionally have to work with multiple clients at once. This means you have to do a lot of context switching. I have had days where I worked on a Power BI sales report makeover for Client A, helped a coworker with some DAX over lunch, and built a data warehouse with product inventory data in Azure SQL DW in the afternoon for Client B. Clients don’t really think about what we do when we aren’t working for them. They are expecting us to mentally pick right back up where we left off when we get back to them and remember all the project and organizational details. I rather enjoy that variety, but I know that it gets stressful and isn’t a good fit for everyone.

Another aspect of project-based work is the amount of time you get to stay on with a client. Some consultants focus on implementations or health checks and then move on. They aren’t there 6 months later to see how things are going with that new system they implemented, unless the client calls them back. There is good and bad in this situation. The consultant may get a lot of experience with building systems/performing health checks in a short amount of time compared to an FTE not in a consulting role. But they may not be around to understand and experience the consequences of their design, unless they client calls them back. You can always work out retainer or support agreements to hedge this a bit, but it is up to your organization and the client to do so.

Project-based work can also mean you have less stability in your life. It’s difficult to anticipate exactly when an engagement will start or end, when you will need to travel, which nights you may need to work late. But project-based work can also offer exciting opportunities, a great variety of technical work, and the opportunity to learn about different companies and industries within a short amount of time.

What would work best for you?

There isn’t a single right answer to these questions. It’s about what would give you the income, benefits, lifestyle, and type of work that you want.

Consulting, Data Warehousing, Microsoft Technologies

Ten Ways To Help Your BI Consultant Be Successful

I’ve been working in the field of business intelligence for over ten years, as a consultant for over five years. One thing I’ve learned from that time is that consultants need the client’s help to complete a project on time and on budget. Even if the consultants are doing the bulk of the work, project owners and stakeholders have a large impact on the project.

When you hire a business intelligence consultant, both you and the consultant want to see your project succeed. While a good consultant can guide you through important decisions and manage a BI project in addition to doing the technical development work, they need your help to get the project started off right and to continue to meet deadlines and requirements. A BI project requires collaboration between the consultant and the client. It’s usually not the type of thing you can throw over the fence and get back a satisfactory solution. We need to understand your business and how you think and talk about your data in order to give it meaning and make it accessible in a data model or report.

In my experience, we consultants write assumptions and project prerequisites into Statements of Work (SOWs) and mention them on planning calls, but we don’t always emphasize how important they are to project success. And we’ll often work around missing prerequisites to try to keep to the project schedule as best as possible, to varying degrees of success. As a client, your organization has allocated budget to your BI project that could have gone to other priorities. We understand this and are motivated the use that budget to accomplish your project goals, but we often spend a lot of project time overcoming obstacles related to lack of access to environments and technical assets and lack of client/stakeholder involvement. The problem/opportunity is already challenging or you wouldn’t need a BI consultant, so why not do what you can to remove barriers that are within your control and help steer the project toward success?

With the help of a few co-workers/work friends I’ve compiled a list of ten ways you can help your BI consultant (and therefore your BI project) be successful. Special thanks to Josh Roll, Melissa Coates, and Levi Syck for your input and feedback on this list.

  1. Have your data sources ready before you start. A good consultant can get started with design and stub things out or use fake data, but it will take us longer and quality will be questionable until we get our hands on real/realistic data.
  2. Work out data access (network/Azure/Power BI logins, VPN access, database access, etc.) for your consultants ahead of time, not on the day of project kick-off.  So many projects get stalled at the outset because the consultants don’t have access to the data and environments they need.
  3. Help your consultant understand any political/departmental boundaries too.  If you know that some department owns some needed data and that they are possessive about it, be up front and consider ways to get them involved, rather than leaving us to go and blunder through, possibly stepping on toes in the process. Provide context for the project. How does it help your organization achieve its larger goals? Who came up with the idea? Has something similar been tried before? Consultants get to do similar projects at different companies, so they bring good experience and ideas for overcoming technical and organizational challenges.
  4. Make sure you understand the time commitment of a BI project and make sure project owners, technical contacts, and subject matter experts can be available as needed. Be involved throughout the project, but especially during user acceptance testing to ensure our solution covers your use cases.
  5. Be able to define success criteria. You may not be able to dictate all the business and technical requirements, but you should be able to work out what success looks like on your project. Your consultant can help you define success, but things will go better if you have given this some thought beforehand.
  6. If you have existing database or ETL frameworks or naming conventions you would like to be used, make sure they are documented, or make someone available during the first few days of the project to explain them and answer questions. Don’t leave your consultant to guess.
  7. If your consultant sends you project planning and requirements documents up front, rather than after the fact, they are using these documents to establish understanding and agreement. Take the time to read them and ask questions. As consultants, we have a limited amount of time to become well acquainted with your data and use-cases, and we operate under the assumption that you will steer us in the right direction if you see us veering off the path.
  8. Be aware of your data hygiene. If your data is incomplete or dirty we’re going to need your help deciding how to handle it.
  9. Plan for an iterative development process. Know that everything probably won’t be perfect the first time. We probably can’t fit everything into the initial scope. Make sure there is room in the timeline for testing and rework. Generally, iterative projects have a higher chance of success than very large big bang projects. You can still get to the larger vision, just know that we will probably ask you to break it up into smaller, more manageable chunks. Also, be prepared to make decisions in the face of ambiguity.  Not all architecture and design decisions can be made with absolute certainty. But they often need to be decided to move forward and can be adjusted down the line, if necessary when priorities change or new information comes to light.
  10. Identify who will support the solution after the consultant is gone. Involve that person or team early. It’s better for the support team to learn about the solution over a period of weeks or months, rather than cramming everything into a knowledge transfer session and a document at the end of the engagement. If you don’t have anyone to support the solution, be honest with yourself and request a separate support contract up front and factor it into budget requests or allocations.

I hope you’ll find this list useful in planning your next engagement with a BI consultant. If you are a BI consultant or have worked with a BI consultant, please leave a comment about what you would add to this list.