How We Designed Our Developer Experience Survey
How satisfied are developers with the Braintree developer experience?
To answer this question we designed a survey that’s short, clear, and to the point. We want to share some of the survey questions, the reasoning behind them, and how we’re building the survey experience.
In the last few months, we’ve been researching how engineers experience our developer documentation. We know we have work to do. We’ve stacked the backlog chock-a-block with all the things we saw builders like yourself hating and with more of the things we saw you loving. We have plans, story points, OKRs, roadmaps, and so many more things to keep us busy this year.
We’re just missing one thing: a tangible way to know we’ve improved. How satisfied are developers with the Braintree experience?
Creating the survey
We designed a survey based on a critical constraint: collect enough data to allow statistically significant insights. This constraint informed a set of design criteria.
Design criteria
- Get value from the first click. Users drop out of surveys for many reasons. By ordering the questions by importance and capturing every interaction using an analytics tool, we can turn an abandoned survey into one with some value.
- Ask the fewest possible questions. This criterion served as a forcing factor to only ask questions that added to our overall measure: developer satisfaction.
- Treat each question as critical. It’s tempting to grab your users’ attention and then try and stretch it out as far as possible. With thoughtful design and copywriting we can respect our users’ time; hopefully, they’ll give us their attention next time, too.
- Don’t be annoying. Enough said.
The questions:
Out of the five questions, we wanted to include both satisfaction questions as well as segmentation questions that make the data useful to not only our direct team but the business as a whole.
Our three segmentation questions are:
- Sales: how big is your business?
- Behavior: what are you busy with right now?
- Product: pick the top 3 things you value in integration documentation.
Our two satisfaction questions:
- NPS style rating: How likely are you to recommend Braintree’s integration docs to other developers?
- Likert scale rating: How satisfied are you with our developer documentation so far?
We then ordered these to collect the information in order of importance. Collecting the NPS first, then understanding more about what users care about in our product, how closely we are tracking to their expectations, what phase of integration they are in, and finally, their business size.
We’re excited to learn more about our developer audience and how well we’re doing through this survey.
Stay tuned for follow-up posts about how this survey performs in the real world and how we’re going to analyze the results by subscribing to our publication.
