Stockgrok Design

After four months of research, we had so many ideas we wanted to get down on paper and try out. We began with the process of “walking the wall,” annotating our affinity diagram with design ideas, questions, and breakdowns. We voted on our most promising ideas as a group, and put them on an impact/feasibility matrix to better understand which ideas we could make happen, and how powerful they might be.

Generating and narrowing down design ideas

For our first foray into the design process, we “walked the wall.” In walking the wall, the entire group, first silently, then with conversation, went through the affinity diagram in depth and write design ideas, questions, and breakdowns we notice from the organized clusters. In narrowing down from our large list of design directions to a more manageable set we planned to prototype, we plotted the most promising ideas on a two-by-two matrix, using our best judgment as to the feasibility of building the idea, as well as the impact it could have on the workflows of people with visual impairments.

Three long white sheets with a white label on top that says 'Design Ideas.' The first sheet lists dozens of design ideas, the second sheet lists 'breakdowns', and the third sheet lists 'questions.' Text too small to read.

Many of our ideas, questions, and breakdowns

This is a picture of the impact/feasibility matrix. Low impact ideas are on the bottom, high impact ideas are at the top, low difficulty ideas are on the left, and high difficulty ideas are on the right. The ideas with the highest impact include a conversational user interface and a querying language to ask questions about security charts. Text too small to read.

Prioritizing our design ideas

Once we had narrowed down our list of ideas to explore, we did an early round of quick prototypes. Rapid prototyping is a way of quickly testing ideas at a low fidelity and low cost level. Often constructed using paper and pencil, this stage of prototyping is useful to get quick feedback on the feasibility and usability of ideas. In our case, since we were primarily dealing with non-screen-based systems, we used the “Wizard of Oz” approach to many of the prototypes; in other words, we would play the role of the computer to simulate more complex interactions with a system. Most of these were then tested with sighted employees at Bloomberg to work out basic usability issues and better understand people’s expectations of our systems. We learned a lot about the infeasibility of some of our ideas, but were happy with our findings with regards to sonification and navigation methods.

Testing early prototypes for usability, feasibility, and desirability

We tested our prototypes with sighted employees at Bloomberg. It made sense at this stage to not worry about testing with people with visual impairments, as our goals at this point were to understand the feasibility of ideas, get a better hold of what kinds of expectations people would have of such systems, and work out early usability issues. We felt that it would be best to wait for higher fidelity prototypes before testing with people with visual impairments given our time constraints and budget.

Each prototype taught us something new. For example, we explored conversational user interfaces (CUIs). In this prototype, there was a big mismatch between what we felt we could provide (“what is the price of Google stock?”) and what users expected we could provide (“compare Apple and Google stock.”) Another promising prototype that subsequently fell into the infeasible category was generating crowdsourced or machine learning descriptions of charts. We learned that the technology isn’t really there, from an ML perspective, and that crowdsourcing won’t work well, especially when taking into account our research insight, “People who help me are interfaces, too.” There was simply no way to determine if a description was accurate.

A man has holds a yellow board with pegs. Rubber bands are wrapped around several pegs forming the zig zagging line of a chart.

A tactile chart with pegs and rubber bands

White foamcore with a grid of holes, with a diagonal line filled in with straws

A tactile chart with discrete points

A simulation of a haptic system

Finally, we explored touch-based interactions. We received very positive feedback about these prototypes, but upon exploring technical implementation, we realized that our examples were very basic, and the cost to provide dynamic, real-time tactile feedback would be prohibitively expensive. Not all of our prototypes were dead-ends, however. We explored using pitch to convey different values within the highly-visual candlestick chart, and found that people were pretty good with keeping track of what the pitches indicated. We also tested different methods of navigation, including a system of knobs to quickly traverse data. People were happy with this intuitive way of speeding up linear navigation. Both of these prototypes made it into our next stage of iterative design.

Takeaways from early prototypes

We had a lot of takeaways from our early design phase. Most notably, this stage was very useful because it helped us figure out which promising ideas would not be feasible. We had a strong notion that users would expect our systems to be very powerful. We also learned that people could do pretty well with identifying different sounds in terms of pitch or timbre. Finally, we learned that some technologies are simply not ready to handle the different interpretations people may have of a visualization.

By the end of our rapid prototyping, we had narrowed down to the medium of sound, the platform of desktop, and the financial task of analyzing security price charts to predict future price movements. While our earlier prototypes had explored aspects of sonification and navigation, it was time to prototype audio and control interactions within a cohesive system. Through our learnings from usability testing of different kinds of auditory output, navigation modes, and task flows, we arrived at the current iteration of Stockgrok.

Usability Testing

Earlier tests of navigation mode and audio cues with eight people with visual impairments helped us narrow down to a solution that used pitch, timbre, and earcon audio cues to provide access to data through keyboard controls. We used these findings to build and refine the higher fidelity Stockgrok prototype through usability tests with two more people with visual impairement. However, because they weren’t investment professionals, we also tested with two sighted domain experts to see if they could make investment decisions using Stockgrok through audio alone with access to the audio equivalent of visual cues used in charts.

	Navigation prototypes	Stockgrok system
People with visual impairments	8 users	2 users
Sighted finance domain experts	0 users	2 users

Navigation prototypes

Bloomberg Keyboard

Stock photograph of a hand holding a stylus up to a drawing tablet

USB Drawing Tablet and Stylus

A palm-sized round knob with a blue border sits on a table

USB Powermate Knob

We tested three navigation modalities: scrubbing with a drawing table, a knob, and keyboard shortcuts to navigate our interfaces. The drawing tablet showed by far the lowest accuracy and lowest user-reported confidence and control. We tested an additional five people with visual impairments with the knob and keyboard. Knob and keyboard were very close on measures of accuracy and user-reported control and confidence. Drawing upon our insights, because the keyboard required no additional hardware and fit into existing finance workflows, we scoped to keyboard.

Sonification findings

We tested several things with sound. We initially used stereo, playing pitch-mapped closing price values in one ear and earcons (audio icons) of two varieties in the other ear to indicate when a stock closed above or below a moving average. We thought stereo could serve as a way to communicate more auditory information at once. We found that users were able to easily pick out the earcon cues, and could do so even without needing both ears. Since requiring both ears cuts the user off from their surroundings, we decided that mono audio would be enough and enable users to multitask, for example listening to colleagues in one ear while using the audio interface at once.

We were also interested to discover that users, particularly sighted users, struggled to understand relative pitches and distinguish timbres (qualities of sound). However, when making the timbres dramatically different, other users noticed the sounds were distractingly abrasive. As we iterated, we tweaked the sound design so that the timbres were more distinctive while also pleasing to the ear. In terms of pitch, we experimented with different pitch-mapping systems. Initially, the pitch had a very wide range so as to allow auditory perception of small changes in values. However, we found that very low and very high values became hard to hear. Not only that, but users still struggled to understand relative pitch values. Because of this, we narrowed the pitch range for all datasets.

Despite variance in the abilities of users to distinguish sounds, think aloud comments and reflective interviews showed that the audio did serve to communicate high level trends, like a general upward price trend for a range. While confidence for using pitch alone was highly variable, when combined with the ability to get details on data with speech at any point, visually impaired users reported high confidence.

Our findings around sonification and navigation, synthesized with research and observations with finance specialists, guided us to build out the final features of Stockgrok. In order to evaluate its effectiveness, we tested it not just with people with visual impairments, but also with sighted finance experts. We wanted to see if people used to visual cues in stock charts could get the information they needed through audio alone while using Stockgrok without visuals.

Task

We presented users with the following task:

"Imagine you're trying to decide whether or not to buy or sell a stock. Use Stockgrok to explore the data and tell us what you observe."

After using Stokgrok without visuals, we asked sighted domain experts to draw the simple moving average and price lines. Pictured below are two sketches experts drew. After a few minutes of training, both sighted experts were able to easily observed the overall upward price trend. Second, both experts also mentioned and drew out the major cross in November, using it as a cue that signaled a new trend. Third, both experts remarked on the jump in magnitude at the end.

As one user clicked through the days, he described the mental image of drawing the two lines in his head: “I can [mentally] draw this out: higher, higher, higher, higher, way extended above the 50 day.”

A sighted finance specialist correctly recreates both axes, the general trend of both price and SMA lines and cross.

A sighted finance specialist recreates the general trend of both price and SMA lines, cross, and jump in price.

We also tested with people with visual impairments who had some previous experience with finance and data. We walked them through the features and concepts of the prototypes. They caught on quickly. Reactions were overwhelmingly positive. They were able to identify many of the same cues as finance experts, such as cross points that signaled a new trend.

One user said: “I can almost see the data by hearing it. It would just take exponentially more time to analyze the data [with a table].”

Overall, these tests validated that experts were able to access the key cues relevant to investment decisions, and people with visual impairments were able to confidently access information from visualizations that was previously inaccessible.

Try Stockgrok for yourself.

Our Design Process

Idea Generation