The Complete Guide to Internal Developer Portals

The Rise of Internal Developer Portals

Today’s engineering teams are under more pressure than ever before. The rise of cloud-native and the growing availability of AI-driven development tools have led to increasingly crowded markets across all industries, fueling the need to deliver innovation quickly and keep customers engaged. To stay competitive, engineering teams must balance speed and agility without breaking things along the way.

Internal Developer Portals (IDP) have emerged to meet this challenge. This guide will help you understand how an IDP can be used to benefit your organization, and who will benefit most. We’ll also discuss the key features of an IDP and how to evaluate them when you’re determining how to best foster high-performing and effective development teams.

What is an Internal Developer Portal?

First, let’s start with the basics. At its core, an IDP serves as the definitive engineering tool for creating, measuring, and operating high-caliber software. IDPs sync all of your tools’ data in a single location, helping to simplify the complex tasks of ensuring software security, maturity, production readiness, and more.

Additionally, internal developer portals often incorporate features for tracking project progress, bringing visibility to dependencies, and automating processes (e.g. continuous integration and deployment).

Although IDPs are relatively new in the DevOps tool space, rapidly growing organizations are already using them as a “force multiplier” to amplify the development capabilities of teams, increasing output without increasing headcount. Given all of these benefits it’s easy to see why IDPs are quickly beginning to play such a crucial role in ensuring a successful environment for software development.

Internal Developer Portal vs. Platform: Understanding the difference

You may sometimes hear the terms developer portal and developer platform used interchangeably, but there is a distinction. A developer portal is not the same as a developer platform. Platform has become a broadly used term, and could refer to tools such as OpenShift, Datadog, or Sentry. And while those tools might be connected, your developers likely have to hop from one to the other or hunt down access to these tools to get complete service information.

A developer portal brings all those things together so that if you're working on a service, you have a unified view of all of the tooling connected to that service and can easily access the info you need, eliminating the need to context switch and freeing up more developer time. Developer portals also facilitate catalog creation and maintenance, software standards, and provide developer self-service resources.

Who Benefits from an Internal Developer Portal?

Internal Developer Portals benefit multiple roles across engineering organizations. Not only do IDPs help to support these roles individually, but they also bridge communication and collaboration gaps, helping to prevent siloes.

• Site reliability engineers (SREs) (AKA DevOps engineer, automation engineer): SREs are responsible for monitoring and maintaining the availability and performance of all services in production. IDPs offer them a centralized portal to understand the reliability and performance of software systems, and the ability to view and collaborate across teams in a single location when troubleshooting an issue. SREs can also run diagnostics and automate tasks to streamline their workflows, improve response times to incidents, enhance system stability, and ultimately ensure a higher level of reliability and availability for the organization's software systems.

• Product engineering team leaders: IDPs provide engineering leaders (Directors, VPs, CTOs) with increased visibility into every aspect of engineering processes, helping them make informed decisions regarding efficiency, prioritization, and troubleshooting. The self-service support provided by IDPs help development teams to work more efficiently, leading to a better developer experience therefore reducing burnout and churn. Additionally, IDPs provide a central location to access tech docs, APIs, and other valuable resources that improve the onboarding experience for new team members.

• Developers: Developers benefit from IDPs as they provide centralized access to documentation, APIs, SDKs, tools, and resources needed for their projects, fueling developing self-service and increasing productivity. It streamlines their workflows, better-equipping teams to adhere to software standards and maintain good service maturity.

• Platform engineers (AKA infrastructure engineer, DevSecOps engineer): Platform engineers are responsible for provisioning engineering teams to ensure compliance with standards, productivity, and deployment consistency. These roles often benefit from IDPs the most as they provide them with a centralized hub for managing and accessing essential tools, services, and resources needed for development, deployment, and operations. IDPs also streamline the onboarding process for new engineers.

Essential Pillars of Internal Developer Portals

The primary purpose of an internal developer portal is to facilitate collaboration, streamline workflows, improve productivity, and ensure standards across development teams. This is achieved through three distinct attributes: visibility, standards, and self-service.

Visibility

Visibility just might be the most important building block to software development. Without complete insight it is nearly impossible to consistently and accurately track the behavior, performance, and health of services. This information is crucial to making informed development decisions. The first step to gaining this insight is creating and maintaining a rich and up-to-date software catalog.

Traditionally, catalogs have been maintained by an individual(s) in a spreadsheet. This method is often error-prone, outdated and simply unsustainable. Additionally, these catalog spreadsheets aren’t always easily accessible to everyone who needs them, often leading to multiple, out-of-date versions being created and maintained to various degrees of accuracy. The most effective way to build and maintain a software catalog is through automation.

How an Automated Service Catalog is Built

Integration: Your IDP integrates with tools commonly used in the software development lifecycle such as Kubernetes, GitLab, Datadog, Jira, PagerDuty, and others. These integrations allow the IDP to gather data about services, microservices, and applications from these platforms.

Discovery: An IDP with automated discovery finds services based on the repositories, projects, or other entities within the integrated tools. It scans these sources for information including code repos, issue tracking, deployment pipelines, monitoring configurations, and more.

Metadata Collection: Metadata is gathered about each service that is discovered. This metadata may include the service name, description, owner, team responsible, dependencies, deployment environments, SLAs, SLOs, documentation links, and other relevant information.

Categorization and Classification: Services are categorized and classified based on various criteria such as technology stack, domain, criticality, and ownership discovered in the metadata. This helps in organizing and managing the service catalog effectively.

Automated Catalog Maintenance: The IDP should continuously monitor for changes in your integrated tools and update the catalog accordingly. For example, when a new repository is created in GitHub, it will automatically detect it as new and add it to the catalog.

Customization and Extensibility: Your IDP should allow customization and extensibility to meet the specific needs of different organizations. Teams can define custom metadata fields, workflows, integrations, and policies to tailor the service catalog to their requirements.

An automated service catalog is the first step in taming the complexity of managing your microservices and plays a critical role in tracking ownership.

‍

Service Ownership

Service ownership refers to the individual or team responsible for the lifecycle of the service. This can include the designing, building, deploying, monitoring, and maintaining of the service to ensure stability, reliability, security, and performance all the way through to EOL (end of life). Service owners are the primary point of contact for their services in the event of an incident or any other development work that “touches” their service. Tracking this information as part of your catalog is key to having all of your service information in one place to quickly identify ownership without having to ping individuals over Slack or through email and facilitate frictionless collaboration.

The YAML Approach

Users may also opt to use YAML files to create their service catalog. Although YAML files can contain more granular data, the creation and maintenance of these files is a manual process and can take months or years to build a catalog as opposed to the days or weeks it can take using an automated process. Additionally, since YAML files are manually created, they are prone to human error and may not even account for all of the services found in an ecosystem. Many users will opt to use a combination of both automated discovery and pre-existing YAML to build their initial service catalog.

Measuring and Setting Software Standards

Standards are established guidelines and specifications that define how software should be built, maintained, and perform. These standards can vary from organization to organization or from team to team based on their own unique criteria.

An IDP enables teams to set, communicate, and adhere to standards while also measuring and monitoring service maturity based on those standards.

Software standards should be:

• Consistent: Apply uniform rules consistently across all aspects of the SDLC.
• Measurable: Include quantifiable metrics to evaluate compliance and performance.
• Actionable: Provide guidance on how they can be implemented and improved.

Standards set the parameters for achieving service maturity, which refers to the “health” of your services.

Scorecards

Scorecards are a tool used to gauge the health and maturity of services. They define metrics based on user-defined “checks” to evaluate production readiness, performance, code quality and other critical aspects.

How scorecards are used to drive standards:

• Visibility and Accountability: Scorecards offer visibility into the maturity levels of services, surfacing areas that need improvement. This promotes accountability among teams and individuals responsible for maintaining service quality, at times creating some healthy competition amongst service owners to keep their high maturity levels.
• Continuous Improvement: Monitoring key metrics and benchmarks through scorecards fosters a culture of continuous improvement and drives ongoing optimization of services. This helps teams to identify and remedy issues before they grow into larger, customer-facing problems.
• Prioritization: Scorecards provide data and insights that can be used to inform decision-making processes related to resource allocation, prioritization of tasks, and development planning. This enables teams to make informed decisions based on the current state and future needs of their services.‍
• Communication and Collaboration: Scorecards serve as a communication tool, facilitating discussions and collaborations among team members, stakeholders, and leadership.
  ‍

Developer Autonomy (Developer Self-Service)

Now that you have established your standards and are measuring your service level maturity, how do you enable your development teams to achieve and maintain those standards with as much efficiency and effectiveness as possible? Automation is the key to reducing development toil and keeping standards on track.

Within an IDP, developer self-service means unblocking development teams so they can get more done faster by breaking down silos, reducing context-switching, and giving devs everything they need in one place. Platform engineers are often tasked with creating a self-service model that gives developers autonomy over their stack, and IDPs can meet many of those requirements.

An IDP offering true developer self-service should include:

• Automated workflows to reduce developer time spent on repetitive tasks.
• Tech and API documentation that is easily accessible.
• Automated service discovery to reduce time allocated for manually adding new services.

Building vs. Buying Your Developer Portal

When deciding whether to build or buy an internal developer portal, it's important to weigh the pros and cons of each approach. While building an IDP from scratch offers the advantages of customizability and control, it requires a significant investment in time, skills, and resources. On the other hand, buying an IDP solution offers the benefits of speed, support, and cost savings. A vendor-provided IDP can be easily integrated into existing systems and offers pre-built features and tools. This is particularly useful if you are looking to solve an immediate problem for example a large migration or recurring incidents while still getting the long-term ROI. Additionally, buying an IDP can often be more cost-effective than building in-house when factoring in the long-term costs of maintenance and upgrades.

A Cultural Shift

Throughout this guide, we've delved into the world of internal developer portals, covering everything from capabilities and requirements to benefits and best practices. But even the most advanced, first-class IDP can fail without a solid adoption strategy.

Here are a few things to look for when considering the addition and implementation of an IDP:

• Seamless Integration: If the portal fails to integrate seamlessly into developers' workflows, it risks rejection. For instance, if the portal's view of running services doesn't accurately reflect their real status, or if it leads to more context switching and bouncing between tools.
• Time to Value: Developers can quickly become frustrated with a solution if it requires heavy labor and resources to stand up, especially if that onboarding and implementation process is drawn out over a long period of time before seeing any actionable value.
• Automation: An IDP is only as good and the quality of information in it. In order to keep your catalog up-to-date and accurate, automated service discovery is key. This also frees up developer time from having to manually maintain it.

A truly effective internal developer portal requires more than just a set of features and a smooth onboarding process. It also nurtures users through understanding the common goals and shared points of everyone involved throughout the entire software development process and uniting those roles on a path to success.

‍

Ready to start improving software standards, driving ownership, and improving productivity? Request a demo to get your hands on your own account today.