Platform engineers are always looking for better ways to distribute architectural patterns and tools across an organization—balancing flexibility, security, and ease of updates.Â
In this post, I'll share what we did before discovering AWS Service Catalog and how it transformed our delivery.
You'll learn what AWS Service Catalog is, how it helps Platform Engineering teams drive value through security, governance, and best practices across the organization.
Table of Contents
Platform Engineering and Serverless
Over the last 5+ years, I have been part (and still am) of CyberArk's platform engineering group. My goals have been clear: build CyberArk's SaaS services and infrastructure on AWS using Serverless technology, help the organization adopt Serverless, improve governance and build services quickly at the highest quality.
Gone are the days where every team delivers its service in a silo with different tools, practices, and solutions, now all teams (mostly) use the same tools and technologies and reduce organizational waste.
In a nutshell, my group does whatever it can to help the other developers in the organization develop Serverless SaaS services faster and provide a smooth and unified experience to CyberArk's SaaS customers.
Now, this is just the tip of the iceberg. To learn more about my journey in the platform engineering domain and how we built our SaaS platform, check out my blog post, "Cloud Platform Engineering - Supercharge Your Development,"Â and view the video below.
Platform Engineering Group - What do We Actually Do?
Let's talk at eye level - what does the platform engineering group actually do:
We build internal SDKs, shared infrastructure, and internal services (authentication, authorization, unified UI, and more), define Serverless and observability best practices, and build tools that provide our customers with a unified experience across our different services and development teams.
Now, you may ask, what problem does the AWS Service Catalog solve for us?
To answer this, we need to go even further into specifics and understand the tools we build.
We built several tools:
AWS Web Application Firewall rules - These rules are built and shared for security and governance purposes, so all services use the same security rules in their AWS accounts and resources. If you don't know what WAF is, check out my blog post.
Architectural patterns as black boxes - provide proven architectural patterns as reusable black boxes. For example - secure AWS S3 Bucket, encrypted SQS, SQS->DLQ with an automatic redrive mechanism (see "Amazon SQS Dead Letter Queues and Failures Handling Best Practices" blog post) and many more.
CloudWatch log exporters. We ship our logs from CloudWatch to Datadog. These exporters enrich the logs with metadata, and send the logs to Datadog. To learn more about the pros and cons of this approach, check out my blog post, "Export CloudWatch Logs to Third-party Observability Tools with Serverless."
Datadog API keys' generation automatic rotations. Since we send logs from every AWS account to Datadog, every account needs to have a Datadog API key. It's also important to rotate these keys automatically.
Development accounts utilities that help reduce cost: EC2 weekend/night time automatic shutdown, automatic cleaning of failed to delete CloudFormation stacks, orphan resource cleaner and more.
These tools are just a few examples of how platform engineering provides governance, FinOps optimizations and security to other teams so they can focus on their business domain and less on mundane SaaS or cloud-related aspects.
Now, the first example, WAF rules, was given as a Python AWS CDK construct, as Python is our main programming language and CDK is our main IaC framework. You can see more examples in my AWS.com article, "How CyberArk is streamlining serverless governance by codifying architectural blueprints."
However, for the rest of the examples 3 to 5, we provided a GitHub template repository that implemented and deployed the solution to the team's AWS account. Teams would often clone the repo, deploy it on their AWS account and maintain it from that moment onwards.
The Problems with Shared Architectural Patterns and GitHub Templates
The main issue is maintenance: updates and ongoing maintenance.
For CDK, our constructs are versioned and have release notes, so it's less than an issue. However, whenever a major version is out, the teams must make changes and deploy their entire service again with the new CDK code, which is a risk. In addition, when dealing with shared stateful CDK constructs, like a secure Aurora cluster construct, if the platform engineering makes a change that causes the cluster to recreate itself by accident, the data will be lost, which is a production horror story.
As for GitHub templates, now the maintenance ownership falls on the team. They need to solve every new Python version upgrade, CI/CD pipeline updates, and own security fixes, which is a lot of overhead for the team. Remember, we want to reduce their work, not increase it!
Lastly, it's all built for Python users. But the world isn't perfect. I work in an enterprise, and there are some services that use C++, Java, and even Go for various GOOD reasons. They also might not even use CDK due to historical reasons or costly migration costs. Some services existed prior to the platform engineering formation. For them, both GitHub templates and CDK constructs solutions don't help at all.
What if there was a way to solve all these issues, provide value to ALL teams, and reduce their effort?
AWS Service Catalog to the Rescue
Service Catalog allows organizations to centrally manage commonly deployed IT services, and helps organizations achieve consistent governance and meet compliance requirements. End users can quickly deploy only the approved IT services they need, following the constraints set by your organization. - AWS
Platform engineering teams can create a portfolio of products and share them with their AWS organization. Each product is a CloudFormation or Terraform template with a controlled version. Then, each AWS account in the organization can import the portfolio, deploy the products, and create provisioned products. Each provisioned product is a CloudFormation standalone stack.
Remember the solutions we provided as GitHub templates and CDK constructs?
They are classic black boxes that teams can use.
With AWS Service Catalog, we can turn them into AWS Service Catalog products and share them with the platform engineering portfolio.
Now, developers can install them as true black boxes with version control. The developers select the version, deploy it on their AWS account, and enjoy all the advantages of using a proven architectural pattern without maintaining a GitHub repository or code. Updates are done via Service Catalog too.
Lets view the console experience to get a grip on these concepts.
The Experience
It all starts with the platform engineering portfolio:
Then you need to share it with other account and they need to import it. See video below:
Once imported, you can see the portfolio's types of products—for example, WAF rules, Datadog logs exporter, and more.
Then, you select the product you want, it's version and create a provisioned product:
Once deployed, you can see all the resources.
Remember, you can do all of this with CDK or other IaC frameworks without requiring manual work. Luckily, I have created a GitHub repository with these code examples, so feel free to deploy and extend it: https://github.com/ran-isenberg/governance-sample-aws-service-catalog
Now that we understand the mechanics let's discuss the pros and cons.
AWS Service Catalog - Pros and Cons
Pros
Teams don't maintain code or GitHub templates. Upgrades are easier and decoupled from their service CI/CD pipelines as each provisioned product is deployed as its own CloudFormation stack.
Lastly, and this is the big one, it provides value to teams even if they are using other programming languages - it's deployed as a black box in which you can interact with its resources. This capability allowed us, as the platform, to provide value to the team that didn't use the same tools (CDK) or programming languages (Python) as us.
Cons
As platform engineers, we require adoption visibility - how can we know we provide value without it?
Service Catalog doesn't provide visibility out of the box which is a shame. It does not show which AWS accounts deployed which products, which versions, etc. This is critical data for governance and adoption.
But, not all is lost. We were able to build it ourselves. I wrote an AWS.com article ("Serverless Governance of Software Deployed with AWS Service Catalog") about adding visibility and governance with Bill Tarr and open-sourced the solution so you, too, could use it!
Here's the open source repository: https://github.com/ran-isenberg/governance-sample-aws-service-catalog
Another major con is that the teams that deploy the products, while they don't maintain the code, need to handle the observability aspects and understand when things go wrong. It's harder when you didn't write it or have easy access to its code—it's all about documentation and training here that the platform engineering team needs to provide.
Summary - My Takeaways
AWS Service Catalog has helped us deliver value faster to all teams by standardizing deployments and simplifying access to pre-approved tools and architectural patterns.
It has replaced our GitHub templates but did not replace shared CDK constructs or Terraform templates in all instances—there's room for both. CDK constructs remain essential for tightly coupled shared infrastructure, while Service Catalog is ideal for independent micro-services and reusable capabilities.
To explore this further, check out my GitHub repository (https://github.com/ran-isenberg/governance-sample-aws-service-catalog) and try deploying a Service Catalog product yourself!