In anticipation of Ruchi Agarwal's session, "How Does Netflix Build Software to Streamline Organizational Operations", Dehla Sokenou and Lukas Zühl (WPS - Workplace Solutions) had a series of questions about the challenges and innovative solutions Netflix has faced in developing internal tools to streamline operations and increase the productivity of its global workforce.
Why does Netflix develop its own tool landscape even though there are a lot of standard tools? Could you share an example of a specific internal tool that was developed to streamline organizational operations and had a particularly transformative impact on employee productivity or collaboration?
In order to customize the tools based on the unique needs of the organizations, many times companies have to build tools that fit their own specific requirements. While standard tools can offer general solutions, oftentimes custom tools are needed to address unique requirements and challenges due to the scale and complexity of a company’s operations. But that being said, we always go through a “build vs. buy” analysis and take proper informed decisions based on various metrics including cost and efforts before going on the route to create our own tools to address a problem.
Do your custom internal tools only have value for Netflix or are they applicable to many other companies as well?
Occasionally, specialized tools are developed to cater to the distinct requirements of Netflix. However, when tools are created with broader applications that extend beyond the specific needs of Netflix, we make a commitment to either make them open source or readily available to the public. This ensures that these tools can benefit a wider audience. Examples of such tools include Spinnaker, Escrow Buddy, ChaosMonkey, and more, all of which have been made accessible for public use.
Developing custom internal tools for a large organization like Netflix can be complex. Could you discuss some of the key challenges your team faced during the development and deployment of these tools, and how you overcame them? Software architecture challenges in particular would be interesting.
During the process of revamping Netflix's employee feedback composition tool, which historically occurred annually with a month-long open feedback writing window, a prominent issue emerged with the older tool. This issue pertained to a significant performance bottleneck that originated within the database. The cause of this bottleneck lay in the thousands of write queries that bombarded the database as users composed their feedback. This was attributed to the previous tool's auto-save feature, which triggered a write transaction every 5 seconds to store ongoing feedback in the database.
Initially, this functionality functioned seamlessly when the company had only a modest number of employees. However, as the company's workforce expanded, the tool's scalability became an acute problem. It led to severe database congestion, particularly during peak hours of feedback composition, especially right before the feedback window closed for the season. Despite users' positive reception of the auto-save feature, the underlying technology proved to be a hindrance, slowing down the process and occasionally resulting in lost feedback when users returned to the tool later.
To address this challenge during the tool's redesign, I proposed the implementation of two key strategies:
1. Leveraging Browser Cache for Auto-Saving: Storing feedback drafts in the browser's cache to enable auto-saving without immediate database write transactions.
2. Introducing a Messaging Queue: Introducing a message queue system between the server and the database to manage concurrent write requests, preventing database congestion.
In addition to these two strategies, several other enhancements were incorporated:
1. Cache Integration: Integration of a cache layer between the server and the database, reducing the need for constant database visits to fulfill user read requests.
2. Redundant Database Servers: Incorporating redundant database servers to amplify the capacity for handling both read and write transactions, thereby enhancing overall performance.
3. Optimized Indexing: Implementing careful and strategic indexing of database tables to accelerate read requests, while also mindful of potential latency issues associated with excessive indexing.
As a result of these changes, the new version of the tool exhibits significantly improved transaction handling efficiency. This has translated into enhanced performance, marked by notable reductions in latency for fetching and reading feedback queries.
Netflix's workforce is spread globally, which likely introduces diverse needs and requirements. How do you ensure that the internal tools you develop cater to the varied needs of employees across different regions and roles within the organization?
Addressing the diverse needs of a globally distributed workforce when developing internal tools is a multifaceted challenge. To ensure these tools cater effectively to employees across different regions and roles, Netflix, or any organization, might employ the following strategies:
1. User-Centered Design: Prioritize a user-centered design approach. Conduct thorough user research, including surveys, interviews, and user personas, to understand the unique needs and pain points of employees in various regions and roles.
2. Global User Testing: Involve employees from different regions and roles in user testing sessions. This helps identify usability issues, cultural nuances, and functional requirements that might not have been apparent otherwise.
3. Localization: Ensure that the internal tools are designed for localization. This involves adapting the tools to various languages, cultural norms, and local regulations. Icons, symbols, and user interface elements should be culturally sensitive and familiar to diverse users.
4. Feedback Loops: Establish ongoing feedback loops with employees across different regions. Regularly gather feedback through channels like surveys, feedback forms, and focus groups to identify evolving needs and areas for improvement.
By combining these strategies, organizations like Netflix can create internal tools that truly address the diverse needs of their global workforce, enhancing productivity, engagement, and overall satisfaction.
Sponsoring a conference is a terrific way to support and connect with our global community of software architects.
