
The 2006 release of SAP ERP 6.0 predates Docker's initial release by seven years! The two technologies are geared towards different scenarios: think monolithic on-premise server vs. cloud-native microservices. What could be gained by combining the two? Software development teams outside of the SAP ecosystem are years ahead in leveraging DevOps ideas, automated testing/continuous integration, and modern version control tools (i.e., GitHub) to realize tremendous productivity and quality gains. Containerizing ERP would be a considerable step towards accessing these advantages for ABAP development.

Here at Nuve, we developed our product to give customers access to SAP's powerful sales and distribution functions through a modern REST API. Our team was not satisfied with the status-quo ABAP development experience, so we decided to take a fresh look at community efforts to run SAP ERP inside a container. We were excited by the prospect of productivity gains and the new development workflows we could achieve -- if only we could spin up ERP instances in a matter of minutes. Spoiler alert: we got there, and it takes 3 minutes to get a fresh ERP instance ready to go! We created a DevOps workflow around this capability that accelerates Nuve development and will ensure we always deliver a fast, effective, and tested solution to our customers. In this blog post, we explore why containerizing SAP ERP was such a tantalizing possibility, an overview of challenges we faced during our journey, and our take on the future of ABAP development.

## Background

If you're new to containers, check out [Docker's introduction](https://docs.docker.com/get-started/overview/).

We need to shout out to one of the critical technologies that enable our approach: [abapGit](https://docs.abapgit.org/). This open-source software provides a 2-way connection between git repositories (GitHub in our case) and an SAP ERP instance. We use this to install code during initialization and also to upload incremental code changes during development.

## What can we do with SAP ERP in a container?

Our goal is to install ERP once, perform basic setup, and then capture this system's snapshot as a docker container image. This image will allow us to quickly replicate our baseline ERP instance by starting a docker container. Armed with this building-block, we next add our ABAP code into ERP by connecting to our git repositories using abapGit. SAP is now ready for development, automated testing, or any other use we can imagine. Here at Nuve, we have two specific use cases:

## Use Case 1: To Each Their Own (An Instance of SAP for each Developer, that is!)

ABAP developers are often forced to develop code in a development system that has been subjected to untold rounds of experimental changes during previous projects, may be shared by developers on concurrent projects, and is often a shared resource among colleagues who may be working on additional aspects of the same project!

We can all relate to this scenario: everything worked at the end of the day yesterday, but now this morning, you can't reproduce your results. You know you didn't change anything, so somebody else likely made changes that interfered with your work. This problem derailed your plans for a productive morning, and you have to spend time tracking down the issue! At Nuve, we do development work inside ERP running in a container. This practice eliminates uncertainty stemming from unrelated changes being made in shared systems. Developers can also be confident that their results will be reproducible in their colleagues' systems. Since everyone works from a common container image with identical versions, enhancement packs, and base configuration (matched to the customer's requirements), no time is wasted resolving compatibility problems or struggling to reproduce results on another system.

Giving each developer their containerized ERP instances has even more benefits: No risk to testing out experimental changes – it's easy to revert the SAP system to a known state. The SAP instance can be run locally or in the cloud, and each developer can run as many systems as they need. Improvements to the base configuration are easy to share with all developers – pull the latest docker image!

## Use Case 2: Is the Phrase "ABAP DevOps" Still an Oxymoron?

At Nuve, we strive to use software development best practices. When working with ABAP code, we are constrained by the tool compatibility with SAP. Here is our (in our opinion modest) list of fundamental ideas for modern DevOps workflows:

-   Keep the code in a git repository instead of in the SAP database (we use GitHub)
-   ABAP code commits must pass automated tests before they can be merged
-   Developers can run automated tests locally to aid in development (tests should be fast so that developers can iterate efficiently)
-   Test results must be deterministic

After analyzing how we could implement these ideas, we determined that we needed a container that would quickly bring up an ERP instance to build the DevOps workflow we wanted. After some container experiments, we settled on a design that bundles a fresh ERP install with these customizations: We apply basic config typical for a manufacturing company, add several materials and customers for testing. We pre-load abapGit along with a small suite of custom Remote Function Calls (RFCs) that allow us to choose additional software repositories to install via abapGit. We can specify code repositories to install when starting the container or call RFCs to do so after the container starts. We have configured our SAP databases to auto-grow. This is important because we'd like to size the DB's such that they are just about to hit their auto-grow thresholds in order to minimize the size of the final docker image. When our container starts, it automatically starts the database and then the application server.

With this configuration, the container is not only helpful in developing ABAP code, but we can also use it in automated testing. The git repositories can contain more than just code; they can define SAP Gateway services and include test data sets for automated testing.

Our automated testing procedure looks something like this:

1. A developer modifies ABAP code in an SAP container instance. They create a feature branch to hold their work. When the code is ready to be merged, they create a pull request from the feature branch in GitHub.
2. Each pull request triggers (via a [webhook](https://docs.github.com/en/github-ae@latest/developers/webhooks-and-events/webhooks)) an automated test of the latest code. The code to be tested is the main branch's HEAD with the feature branch applied on top.
3. The repository also contains the automated testing configuration. In our case, it describes Kubernetes Pods to start up and a series of commands to run inside of them. One of the containers inside the Pod runs our SAP ERP image.
4. We run a series of RFCs on the SAP instance to instruct abapGit to install the version of the code being tested, along with specific versions of any other repositories that are dependencies.
5. The results of these tests are reported back to GitHub and are required to succeed before the pull request can be merged (check out [GitHub's Branch Protection Rules](https://docs.github.com/en/github/administering-a-repository/about-protected-branches)).

## Docker Challenges

Getting SAP ERP to run inside a container took a great deal of persistence along with trial and error. We developed a procedure to perform an unattended SAP install but still had to keep careful notes of several manual steps of the post-install process. To complicate our effort, Docker's default configuration is not compatible with this use case. For example, Docker assigns randomly-generated hostnames to each container build step, but SAP requires the hostname to be a fixed value. We have to employ some Linux tricks to convince the SAP installation process that our container has the expected hostname.

Another issue we ran into was a limitation on the "base device size" (by default, Docker's dm.basesize is 10GB). Our image turned out to be over 100GB. Remember that you will also have to upload this image to the cloud (we are using Google Cloud Platform's Container Registry). If your internet connection isn't up to the task of uploading 100GB, you may have to run the entire docker build process inside a VM in the cloud.

We also experienced strange behavior getting SAP to start in our completed image – the container would hang for several minutes when starting the database. After hours of investigation, we realized that Docker's default overlay2 storage driver uses file-level copy-on-write. The first time you write to any file inside the docker image, the file has to be copied out of the image so that the local writes can be tracked. The SAP main database file was 88GB, and it was taking so long to copy the file that the SAP start script timed out and assumed something had gone wrong. This problem was tricky to solve. We found an alternative docker storage driver that handled SAP's large DB files more efficiently, but it wasn't supported by Google Cloud Platform's managed Kubernetes service. Ultimately we placed SAP's database files in volumes and used volume snapshots to give each container instance fresh copies of SAP's huge database files.

## Results

Ultimately we built a docker container running SAP ERP. We can spin up a fresh instance in about 3 minutes. Almost half of this time is spent restoring volume snapshots for SAP's DB, and the other half is starting the SAP database and application server. Although we'd still like to reduce the container start time (we believe reducing testing iteration time increases efficiency), this solution enables the DevOps workflows we need.

## The Future of ABAP Development

Although SAP is working towards supporting modern development techniques and tools (S4/HANA express edition is available as a [docker container image](https://hub.docker.com/publishers/saplabs)), it appears that SAP ERP is largely being left out of the official efforts. With years to go before the current 2027 end-of-life target for SAP ERP 6.0, we will be developing new software for ERP for years while customers start to make long-term plans to upgrade. We believe connecting web services through an API like our Nuve product provides an improvement in the customer's experience using their current SAP ERP instance today, with the bonus of offering a clear upgrade path to S4/HANA equipped with Nuve API in the future. The Nuve API remains constant before and after the upgrade, meaning that a simultaneous upgrade of web services is greatly simplified, and the user experience doesn't have to change at all!

The ABAP open-source community is still small compared to other modern languages but is gaining in strength. I believe we can attribute this acceleration to the increased accessibility offered by the foundational abapGit package, which has finally made it practical to share ABAP code online. ABAP development workflows will always lag behind workflows for modern languages like python and javascript, but hopefully we'll continue to gain access to essential tools that currently do not integrate with SAP.

Originally posted on the SAP Community Blog
[Our Journey with SAP ERP DevOps Part 1/2](https://blogs.sap.com/2021/03/26/our-journey-with-sap-erp-devops-part-1-2/)<br />
[Our Journey with SAP ERP DevOps Part 2/2](https://blogs.sap.com/2021/03/30/our-journey-with-sap-erp-devops-part-2-2/)<br />


How We Achieved Tremendous Productivity & Quality Gains

The 2006 release of SAP ERP 6.0 predates Docker's initial release by seven years! The two technologies are geared towards different scenarios: think monolithic on-premise server vs. cloud-native microservices. What could be gained by combining the two? Software development teams outside of the SAP ecosystem are years ahead in leveraging DevOps ideas, automated testing/continuous integration, and modern version control tools (i.e., GitHub) to realize tremendous productivity and quality gains. Containerizing ERP would be a considerable step towards accessing these advantages for ABAP development.

Our journey with SAP ERP DevOps

How We Achieved Tremendous Productivity & Quality Gains

Related Posts

Infrastructure as code, DevOps, and the future of enterprise

Our journey with SAP ERP DevOps

A practical guide to DevOps for SAP ERP