Here are studies from scientists at Merseburg University of Applied Sciences which I summarized in this article:

• Application Programming Interface Documentation: What Do Software Developers Want?, 2018
• API Documentation Optimization: Some Guidelines & Effects, 2020

The studies shed light on the nuanced preferences of developers, offering valuable insights for future API documentation endeavors.

What do developers want?​

In the first study, researchers collected the experiences and opinions of developers to understand their approach to learning a new API: what information they seek, what resources they use, and what hinders them.

Developers were asked to share their experiences with APIs and documentation. Based on these insights, a survey was conducted with other developers to confirm that the information from the initial interviews was relevant and important for all developers.

The research reveals that developers approach new APIs with two different goals:

1. To understand whether to choose the API for a specific task.
2. To use the API to perform tasks.

Regardless of the goal, the first thing developers want to know when starting to work with an API is a brief overview of what the API is and what tasks it can be used for. After that, opinions diverge:

• Quick starters: Developers who want to begin coding immediately. They look for examples that could serve as a basis for solving their problem.
• Understanding first: Developers who prefer to understand the API before starting work. They search for and read API documentation, such as an overview of technical architecture explaining common concepts. They approach documentation based on the problem or task at hand.

Challenges developers face​

Developers often encounter problems when studying APIs:

• Inaccurate or error-filled documentation
• Unclear documentation
• Difficulty finding information (e.g., determining how the API should be used to solve a specific problem or where to start)

If developers encounter a problem, most will Google it, with only 30% reading the documentation.

What to do about it?​

Drawing from the insights of the first study and research by other authors, the second study formulated recommendations for API documentation and tested their effectiveness.

Researchers revamped a real API’s documentation and assigned developers tasks using the API:

• Some developers received the old documentation
• Others received the new, improved documentation

The study measured:

• Accuracy of task completion
• Time spent on tasks
• Time spent consulting documentation and external resources

Results​

• Developers who used the new documentation made fewer mistakes.
• Task completion times were roughly the same.
• Time spent reading documentation did not differ, but developers using the old documentation spent more time consulting other resources, leading to higher overall effort.

Feedback from developers indicated that both sets of documentation have room for improvement.

Conclusions​

The research emphasizes the importance of user-friendly API documentation. Recommendations from the second study align with industry best practices, such as Tom Johnson's Documenting APIs: A guide for technical writers and engineers.

A summary of Twilio's presentation on transforming their documentation. You can watch the full presentation in one of their videos or read it on their blog.

Background​

Once upon a time, Twilio's documentation contained a lot of text that explained the complex concepts of their service and how everything worked. It was well-organized documentation, comprising 1000 pages maintained by 5 engineers. Everything was good, but why not make it better?

Twilio conducted research, and the documentation feedback was positive—developers liked it. However, when developers were asked to show how they used the documentation, it turned out they were scrolling through the documentation to find code examples. They read code examples, took them if they fit, and continued searching if not. After several unsuccessful attempts, they resorted to Google and eventually found the required documentation page. Developers were not using the table of contents and well-structured menus to find information.

Therefore, Twilio began devising code-first documentation solutions to teach developers how to use their service through code.

Tools to Build Code-First Documentation​

For the documentation site, they used:

• Wagtail CMS based on Django: With the StreamField tool, they could combine structured and unstructured content. Twilio uses this tool to embed code examples into the documentation.
• GitHub: They store all code examples in a separate repository and test them as real code. They also receive pull requests from external developers who tried the code examples and know how to improve them.

For analytics and research:

• Optimizely for A/B testing
• MixPanel and Google Analytics for metrics
• UserTesting for video interviews with developers

Side Notes​

A few more discoveries by Twilio about developers' behavior:

• The main thing is to help the developer complete the first step. Developers are more likely to finish a guide if they succeed in the first step.
• The less text before a code example, the better: pages with 4 sentences before a code example performed twice as well as those with 11 lines before the code example.

Welcome to Readme.ltd—where complexity meets clarity.

Our mission is to turn complex ideas into technical documentation that’s clear, concise, and effortless to use.

We believe that great documentation not only empowers users but also enhances the value of your products. Our team is dedicated to creating content that is accurate, accessible, and engaging, so your users and teams can get the most out of your technology.

Stay tuned for insights, best practices, and tips on creating documentation that truly works. Thank you for joining us on this journey!