Platform engineering and IDP

The concept of platform engineering is the step up evolution of setting up deployment automation using CICD tools and utilizing the Cloud provider integration for hosting the apps.
IDP is its further evolution to reduce the dependency of DevOps engineers' interaction with Developers. Because these concepts help to set up a self service platform for developers to onboard their app when they are ready to deploy.

These concepts are mainly driven by the pipeline automation done using Pipeline as a code and creating templates for reusable deployments.

Approaching the platform engineering concept is a step by step process, so let me explain my view of this concept (this might be different from person to person).

Deployment automation using CICD

Step 1: Usual cycle of application onboarding,

It starts with usual deployment automation using CICD,

1. when the dev teams complete developing a feature, there is a need for development teams to reach out to DevOps engineers to onboard their application for promoting the code for the release

2. So before the feature development starts, devops engineers or leads works in parallel with development team to understand the build and app deployment process from dev teams

3. DevOps engineer/lead will come up with a deployment strategy (including the integration with infra automation, test automation tools and monitoring tools) for the specific feature development, that was based on the nature of programming language used typically. In some cases, architecture decisions need to be considered, if the application is fit for container based deployment or non container based deployment.

4. Then using the tools automation knowledge of DevOps engineers pipeline will be created with various integration needed for CI and CD tools

5. Once the pipeline is ready, it is recommended to test the build and deployment strategy with some sample application provided by the dev team, to confirm the strategy works good and it can be used to on-board when the actual application development is complete

6. In case of multiple feature development teams, this process will be repeated for each teams

7. So finally when the feature development is completed, the dev team approaches the DevOps engineer to on-board the application in the pipeline

The process so far explained can be the typical example of how a feature development team utilizes the DevOps engineer team for application onboarding.

Platform Engineering

Step 2: Evolution from Individual deployment automation to Platform Engineering

• Now we can notice there is a repeatable pattern of efforts involved here for feature teams and there is dependency for developers with devops engineers. Here is when the usual deployment automation gets evolved into platform engineering. As I mentioned earlier every deployment orchestration tool available now be it OSS or enterprise tools, supports creating the pipeline as code.

• So to simplify the application onboarding for development teams, DevOps engineers will be creating a pipeline as code and will make it as a template for each specific development team. The template can be re-used onboarding apps of the same programming language.

• Also another advantage of pipeline as code is, it can be source controlled in git so the pipeline will be versioned for various releases too.

• So the process of strategising the deployment automation and templating the deployment will be a major responsibility here. The team responsible for implementing it is as platform engineering team and engineers are called platform engineers.

• This entire deployment automation and onboarding of apps will be documented and handover will be given to developers on how to use this template for further app onboarding. So this reduces the dependency of DevOps engineers for dev teams.

Internal Developer Portal

Then let's quickly discuss what is IDP as well.

• Internal Developer Portal (or Platform) - IDP in short is an extension of the platform engineering team. We have discussed how the deployment automation process was simplified using pipeline as code template, and version controlling it.
• So the IDP platform will be having an UI to wrap all the various existing pipeline templates of specific tech stacks in the portal. Thus it further simplifies the job for developers.
• Typical IDP platform should have customisation options to choose the tech stack and what is required in the pipeline. When the developers chooses the feature development repo, the internal pipeline template will be picked up for the specific tech stack and generate an automated pipeline to onboard the app.
• Also it will contain various dashboards which will show various release metrics for stakeholders to understand how the feature development is going. For typical IDP contain the pipeline health metric which will show how the deployments are performing and how is the health of specific services deployed in infrastructure.

Conclusion

In this blog, I've explained the abstract understanding of platform engineering and IDP in this post. We have not deep dived into each topic. So If you want to know more about the specific topic please reach out to me.

Follow me to share your thoughts,

• LinkedIn
• Twitter
• YouTube
• gsaravanan.dev
• GitHub