The purpose of this post is to help you understand how to scale iDempiere to enterprise levels. I recently gave a presentation on this topic at the iDempiere World Conference, and I wanted to share the highlights with you here. Please note that some of the below tools and scripts are only available to ERP Academy members; however, I wanted to at least discuss the concepts so that you may get started more quickly.
I hope this information helps!!
Starting with Simple
Installing iDempiere for one or two people during an evaluation phase is pretty easy. This page gives you a detailed description of the process. You can use these guidelines to create, test and destroy iDempiere instances with little effort; however, when you decide that iDempiere is a good fit, you will want something a little more stable and robust to support your team’s efforts.
Back Ups Help
Two needs arise when you decide you want to move beyond ‘playing’ with open source ERP iDempiere:
- You need to back up your work
- You need to create two environments: one for your pristine or production configuration and one for testing and destruction.
Before go-live, the pristine environment is held private to you and your core ERP project team. The testing/destruction environment is exposed so that anyone in your enterprise can demonstrate practical real-life scenarios. After go-live, your pristine environment is your production environment. The testing/destruction environment is used to solve challenges before you enter the real data in your production instance. The ERP Academy give you automated tools for creating and keeping these environments in sync and available with little manual labor.
For example, the ERP Academy has an offsite backup script that will create snapshots of your pristine database and move them offsite to S3 storage. It also has a restore script to automatically:
- Shut down your test server
- Load the ‘latest’ snapshot into your test server database
- Start the test server
If you put the above ‘restore’ script on a test server cron job, then every morning at 6AM your users get a fresh copy of the pristine data. If you have not gone live yet, then the test server has a copy of the most recent production configuration. If you have gone live, then the test server represents your production data as of the ‘latest’ snapshot. Auditors love when you practice disaster recovery on a daily basis. Users love when they have a sandbox they can use to solve problems without needing to go to the IT/ERP team. The IT/ERP teams love when users can solve their own problems.
The ERP Academy also supplies scripts to create hot replication of your iDempiere database. If you have many applications that read from ERP data (CRM, Web stores, Business Intelligence, etc…), your read replica will support much of your read only load as well as serve as a hot standby in case of disaster. The ERP Academy helps you create read replicas with a single Linux command.
Choose your Platform
For me, the hosting operating system choice is easy. I choose Linux. Specifically, I choose the current Ubuntu LTS. All my automation scripts and support tools exist in Linux, and Linux helps me keep the average cost per concurrent user down. Here is an article that discusses the iDempiere cost per concurrent user.
Most of my iDempiere hosting experience is with Amazon’s AWS platform; however, I have written all installation and configuration scripts to work with any hosting platform (Local LAN, Rackspace, IBM, Openstack, Digital Ocean, Strato, etc…).
Size Matters – Choose your Stack
If you are putting an ERP into production, stability and scale-ability matter. At a minimum, you want to separate your application server and your database onto two different machines. I have seen this simple setup scale to over 300 concurrent users; however, it is not ideal for organizations with more than 50 concurrent users. If your application server goes down, every head in your office pops up.
A more high availability (HA) iDempiere hosting stack look like this:
- HAProxy load balances between multiple Application Servers (install and configuration script in ERP Academy)
- Multiple UI (WebUI) application servers – create as many as is needed to safely support your concurrent users. FYI – Each end-user session will consume between 150MB and 300MB of memory depending on the user’s role.
- Dedicated services application server outside of the iDempiere HAProxy pool. The ERP Academy installation script makes this a one line command.
- Dedicated PGBouncer connection pooler. The install and configuration script is located in ERP Academy. This is only needed if your WebUI pool grows beyond 4. Keep in mind that the iDempiere connection pooler (found in each application server) comes configured to use between 5 min and 90 max connections. The PGBouncer node helps make memory management on the PostgreSQL database easier to manage and predict.
- Dedicated PostgreSQL database
- Dedicated PostgreSQL read replica – preferably in a different availability zone.
Please note that user sessions are not replicated across WebUI application servers. If any UI server goes down, users on that server will be immediately asked to log back into a different server. In most instances, this short amount of downtime is acceptable.
iDempiere has a loose concept of releases. Even though version 2.1 was officially released more than a year, this release continues to receive many new commits. This situation is great for the project’s continuous improvement; however, it creates instability in company’s release processes if they are not insulted from these changes. This situation will only become more exaggerated over time. The vision is that iDempiere (the project) ceases to create releases. Distributions like JPiere and ERP Academy would instead create releases based on time, features and testing.
If you want to maintain a production quality iDempiere platform, you will need to create your own min-release/distribution or team up with a distribution that does. The informal (not recommended) way to accomplish this task is to create a developer workstation where you clone and build iDempiere from your development environment.
Here is my recommended approach:
- Create a fork of the iDempiere branch
- In your fork, create a branch that resembles your company or project name
- Create a jenkins build server to build your official version of iDempiere
When is a Fork a Distribution
Many people become nervous when you speak of “forks” in an open source project. In the above scenario, a fork is not a fork – not really. Instead, think of it as a copy that will get batch updates instead of daily commits. The value of the jenkins-based approach is that you can insulate your production version of iDempiere from the many daily commits to the core iDempiere project. From time to time, your organization will need to review, test and release a new version of iDempiere with updated features.
My installation script acts in a similar manner. I have my own jenkins build server. My installation script pulls from my jenkins build server. The build server pulls from a local clone of iDempiere. I only update my local clone of iDempiere when I see a feature that is worthy of the effort to perform regression testing. This approach helps ensure I continue to create stable builds of iDempiere.
There are recorded sessions and frequent discussions about this topic in the ERP Academy. Most major companies (integrator and end-user) have their own build server. I am happy to help you establish and automate your build and release process.
Way Beyond Reporting
When most technical people speak of reporting, they think of native print formats or jasper reports. Enterprise reporting goes way beyond any one report based on any one table. Transactional databases do not make for good reporting. Instead, you need a good business intelligence (fact/dimension based reporting platform) to support end user data exploration and visualization.
The ERP Academy offers an ETL script that turns iDempiere’s transactional model into a reporting platform for BI tools as complex as MicroStrategy or as simple as Jasper Server Enterprise Edition. The ERP Academy BI ETL script automates the data warehouse creation in a dedicated server.
Be aware that the iDempiere project does not spend much time on creating performance-based indexes. When scaling iDempiere, you will face database performance bottle necks. Most of the time, a good index will make the world of difference. Tools like PGBadger can help you quickly identify opportunities for improvement.
The single biggest reason I have seen the systems slow down due to customization is because of iDempiere’s ColumnSQL (virtual column) feature. ColumnSQL gives iDempiere windows the ability to dynamically bring in additional fields from a separate table. This is a powerful feature. If your ColumnSQL references indexed data, it is a fast feature; however, be aware that adding ‘heavy’ or slow queries to ColumnSQL will your system down. This fact is especially true on high-volume tables like AD_User, C_BPartner, C_Order, etc…
Eyes on the Prize
Monitoring and graphing/visualization play a key role in providing a high-availability iDempiere open source ERP hosting environment. The most common open source monitoring solution is Nagios. It has plugins for both PostgreSQL and JVM. Datadog is a nicely-featured and well-priced SaaS monitoring solution. The most common open source data visualization is graphite. Another tool for visualizing PostgreSQL performance is PGBadger.
Growth in concurrent users sometimes also means growth in the number of consultants, integrators and developers involved in your project. ERP Academy helps you extend your technology resource pool while keeping your sensitive data private by providing database obfuscation scripts and automation. ERP Academy scripts will periodically back up production data, push the back up to a obfuscation instance, produce the obfuscated copy, and push the copy to a shared s3 folder.
ERP Academy offers a couple of options to keep your iDempiere instance secure.
- Reverse Proxy – script to install and configure Apache as a reverse proxy to help protect iDempiere application server from the outside world
- SSL configuration – Apache configuration to create an SSL termination point.
- Apache firewall – configuration where Apache protects iDempiere’s identity behind a generic SSL-encrypted username password prompt. Apache authenticates directly against your iDempiere user list. This solution is not really a firewall; however, it goes a very long way to ensure no-one knows what is running on the server.
- OpenVPN termination – script to quickly configure an OpenVPN termination point inside your network. This quick and easy solution allows you to hide iDempiere behind one or more firewalls and still give provide iDempiere access to mobile users.
- Full PFSense firewall – detailed instructions to configure PFSense in AWS using public and private facing subnets. This structure gives you almost unlimited options in terms of how you protect and connect to iDempiere. You can maintain multiple site to site VPN connections, mobile OpenVPN connections, BGP routing, quality of service traffic shaping, etc…
Hosting an enterprise quality iDempiere instance requires some institutional knowledge. I help organizations remove barriers and learn, configure, audit, launch and scale iDempiere open source ERP to meet your needs. I focus on teaching and knowledge transfer. I also focus on automating tasks when possible. Many of the topics discussed in this post are automatically available through ERP Academy scripts. I hope this helps, and I look forward to working with you.
What is the best way to Learn iDempiere and ADempiere?
I teach an on-line class that covers how to learn, configure and audit open source ERP. It uses iDempiere as the reference ERP. Here are the course frequently asked questions. I have learned much over the last fourteen years, and I have much to share. I look forward to seeing you there!!
Why consider Open Source ERP?
Open source ERP gives you every opportunity to prove or disprove its ability to support your company’s ERP needs on a timeline that satisfies your organizational needs. With open source ERP, you do not face the same financial constraints nor do you face the same conflicts of interest as with commercial ERP. Instead, you invest in the appropriate skills and knowledge for your people and processes. Best of all – if open source ERP cannot solve your company’s needs, you can safely justify spending the additional $2K to $5K per person per year for life of your commercial ERP to help drive your organization’s success.
Open Source ERP Round Rug Effect
Open Source ERP has what I call a “Round Rug Effect”. If you were to liken the ERP evaluation process to a 10′ x 10′ room, the story would go something like this:
- Oracle, SAP, and Microsoft are a 10′ x 10′ ERP rug in a ten by ten foot room. They cover the room nicely. You will be hard pressed to find a feature or a use case that they do not cover.
- Open Source ERP is like a 10′ round rug in a ten by ten foot room. It will cover the vast majority of the room; however, it will leave the corners bare. The questions are: “Do you live and operate in the corners?” or “Is open source ERP good enough?”. For most, the answers are “sometimes” and “yes”.
If you are in the ERP evaluation mode, you should ask yourself “Should I include open source ERP in my evaluation process?” If you are less than $300M USD revenue, your answer should probably be yes! This answer comes from these concepts:
- Pillars of Cost – Since open source ERP is free, that means that all the cost of proprietary ERP should be allocated to the corners. If you use height to illustrate this allocated cost, the corners turn into tall pillers of cost.
- Cost of Innovation – At first look, the price tag of free open source ERP is the most appealing benefit; however, this benefit soon becomes overshadowed by the flexibility of open source ERP. If organizational leaders take just some of the cost that would otherwise be spent on Oracle or SAP, and they invest it back into the organization’s skills and knowledge of how ERP works, operational efficiency will never look the same again. If you know how to change the system for the better, and you know it will work. Why would you not?
- Monday to Monday Cycle – Business leaders drive innovation in a company. This innovation is no more apparent than in the traditional Monday morning business meeting where a CEO comes in and paints a picture of the next greatest thing. His or her next comments are “Will it work?” and “Make it happen!”. Open source ERP helps your business and IT teams say yes more often. You are no longer completely dependent on a high-priced Oracle Integrators. You are no longer dependent on spending 18% every year to Oracle for software that you have little control over. Your team applies its knowledge of the system and the knowledge of its world-wide resources to create a proof of concept that paints the real picture the following Monday.
- Right Pay Grade – Open source ERP puts the right tools in the right person’s hands at the right pay-grade. there is little more wasteful that paying a $150/hr integrator for something a Jr IT professional should be doing. Open Source ERP removes the artificial barriers that exist in proprietary ERP.
- ERP for Everyone – User licenses/seats are no longer a consideration. This point cannot be stated strongly enough. At first look, you might think this point is about saving money. It is much more than that. You now have the freedom and flexibility of allowing everyone in your company to interact the system that drives your operations. You simply assign the right roles to the right people to give them access to the appropriate information.
ADempiere vs iDempiere vs Openbravo vs Compiere
The ADempiere, iDempiere, Openbravo and Compiere environments are amazingly similar. iDempiere came from ADempiere. ADempiere and Openbravo came from Compiere. Compiere came from Jorg Janke. Jorg came from Oracle. As a result, iDempiere and ADempiere have much in common with Oracle’s ERP in terms of the financial feature set.
This is both good and bad. Good because iDempiere and ADempiere are quite capable to help a company grow beyond $500M USD. Bad because they tend to be more complex in that they account for multiple languages, accounting schemas, currencies, calendars, costing types, costing methods, etc…. If you are a growing organization, and you need a system that will grow with you, and you have the right internal talent/resources, iDempiere or ADempiere will be a big asset for you.
The biggest difference between these products is that ADempiere and iDempiere are pure open source. ADempiere and iDempiere make all feature available for free. Compiere and Openbravo hold back features behind a commercial or paid license.
Here is an article that discusses the differences between iDempiere and ADempiere.
iDempiere and ADempiere vs Odoo
iDempiere/ADempiere (iD/AD) and Odoo (formerly OpenERP) approach ERP from two very different directions. Odoo comes out of the box with very simple options. If you are coming from QuickBooks, and you need a simple ERP system help you manage your business, Odoo will look and feel comfortable.
iD/AD comes out of the box with every feature installed and configured to run a $200M+ USD business. If your business is growing rapidly, and you are willing to invest the time to learn an enterprise accounting system, then iD/AD will give you confidence.
Which one is best for you depends on your internal talent, growth and business complexity. Here is a post to help you learn more.