A couple of days ago I wrote my first blog post about my experience at the Magento Developers Paradise 2011 on Ibiza and promised to dig deeper into the details of Magento 2, which I noted while listening to the talks by Yoav Kutner, Dmitry Soroka and Anton Makarenko. I think that this project is crucial for Magento to keep and develop its most precious asset: the Magento community. The Magento 2 project will show what the acquisition of Magento by Ebay really means for us. The release date is planned for mid/end of 2012.
Transparency
The first thing I want to mention is that Magento wants the development process be more transparent to the community. Therefore Magento will soon provide view access to the development tool stack, which consists of following Atlassian products:
- Jira – issue and project tracking
- Confluence – wiki for documentation
- Bamboo – continuous integration
- Crucible – code review tool
- Fisheye – repository browser (thanks for the screenshot by Andreas von Studnitz in his blogpost about Magento 2)
Jira should give us a much better overview of the currently developed release and the planning of further releases. Confluence will provide a wiki with the Magento documentation. Bamboo should show metrics like broken builds ratio, unit tests count and code coverage percentage. Making Crucible public will provide a completely transparent code review process. I can’t really believe that this will be made public, because it is very intimate ;). And Fisheye will make browsing the code and looking on diffs between version smoother than now (actually you can only browse the directory tree in SVN and just look into files).
It was also interesting to hear that the Magento core team consist of 6 developers and is using Scrum.
Software Quality
Magento was and is often critisized that Magento lacks of automated tests and proper documentation. And in deed this leads to unstable releases and code which is hard to understand. At the moment maintaining Magento solutions is more expensive as it should be. The time to qualify a PHP developer to be able to work with Magento is quite long (approx. 6 months which is not only my opinion) and expensive. This makes it harder for Magento solution partners to scale their development capacity. In addition, the lack of automated tests makes it harder to develop and maintain customizations and even the core itself. Clients expect a stable solution and low maintenance costs from their Magento solution partner. Magento 2 is supposed to address these issues:
- there are already approximately 1900 automated unit, integrity and performance tests for Magento 2
- they already cover 40-50% of the code
- the documentation process (inline and also external in Confluence) is implemented in the development process of Magento 2
- there won’t be features which are released without automated test coverage and documentation
- Magento tries to cut the time to qualify a developer by 50%
- Magento considers porting the tests down to Magento 1 (I’m doubtful about this one)
This is nice to hear, but I recommend Magento to fulfil these announcement for Magento 2. I remember when I have visited the Magento office in Kiev for the first time in the mid of June 2009. I’ve discussed lots of topics with the team around Dima, Micha and Anton and we also talked about unit tests.
I'm the guy on the left, Anton, Micha and Dima from Varien from the right (2009)
A couple of weeks after the trip I saw this tweet: http://twitter.com/magento/status/2506514713 and was very excited about this news, but still we don’t have automated tests today.
The external documentation in Confluence is also a very good point. There is no official publication about the Magento architecture in form of a book. The first literature I’ve discovered was a book written by Mark Kimsal in 2008: php/architect’s Guide to E-Commerce Programming with Magento. It covers some basic architecture and digs into certain important modules. This was not really it, but it was better than nothing back then in 2008. In 2010 O’Reilly has published a German book called “Magento – Das Handbuch für Entwickler“, which I have co-authored. It went deeper into the core architecture and provided much more examples of how to apply the knowledge to practical tasks and issues. Unfortunately analyzing the book reviews today show, that people still have issues transferring the theory into real Magento development. Until now I haven’t heard of any other books which go deep into the core of the code. Most of the books concentrate on the installation, configuration and guiding admin users through the Magento Admin Panel functionality for the shop setup and the daily business of a Magento store after the Go Live.
Long story short: I’m very excited to see the official Magento documentation and hope that developers can benefit from it and that it can be used to educate Magento developers.
Frontend
I’ve enjoyed hearing that Magento 2 will throw out Prototype and extJs and will use jQuery only. My team at next commerce recently completed a project which target was to create a frontend theme which is using jQuery only and doesn’t contain any inline Javascript and CSS in the templates. It will be a component of our D2C software core and will be used in future D2C projects.
Magento 2 will have unlimited theme fallback: this means that you can build an unlimited theme inheritance tree which allows avoiding code redundancy in the templates, CSS and JS files completely. In Magento 1 there are only three levels of inheritance.
Example for multiple theme inheritance in Magento 2
A GUI design editor will provide functionality like moving blocks in layouts via drap&drop and it also will be able to export pages as one template file which will consist of all block templates and all states.The designer can edit that template instead of having to perform changes in lots of templates scattered in lots of different modules.
Architecture
Since Magento 2 is first of all a refactoring release, there will be a lot of changes to the architecture to improve scalability, extendability and quality of Magento.
In Magento 2 there won’t be any magic getters and setters. This means that the interfaces to all objects in Magento have to be strictly declared in the code. Also the data model definition will be declared, so that the database is no more the source of the data model definition of objects (this will actually have an impact on EAV). Together with the stricter documentation policy we should have finally a proper API documentation for Magento 2.
Currently, Magento objects like products, orders and customers are contaminated with things like controlling the persistance of their data (load, save, delete). There will be a strict separation of domain models (define which characteristics objects like products have) and service models (perform operations on objects).
If you install two Magento extensions which are rewriting the same model, block, helper, etc. from the core, one of the extensions won’t work, because the first extensions wins when it comes to rewriting. In Magento 1 the only solution for that is to modify one of the modules to build an inheritance chain, so that the modules both work again (of course this only works if the extensions change different characteristics of a class). Magento 2 will allow to configure inheritance chains of models from additional extensions without code changes, which makes it easier to upgrade these extensions. This is a feature which OXID eSales already has.
To be able to disable functionality easier, Magento 2 will have less coupling between modules. Magento 2 introduces components, which consist of tightly coupled modules: Sales (Sales, Checkout), Catalog (Catalog, CatalogIndex, CatalogSearch, …), CMS. These components can be disabled without breaking the rest of the system, so it will be possible to disable Sales to have just a browseable catalog and static content (CMS) or just only static content on the site, if you disable Sales and Catalog.
Modules in Magento 2 will contain all files which belong to the module in one folder: app/code/<pool>/<space>/<model>. Not only PHP classes, but also the templates and skin files will be in this path. The filesystem structure will be rearranged: there will be a folder called “pub”, which will contain all files which should be accessible to the public (jslib, media, skins). The other directories (app, dev, lib, var) should not be delivered by the webserver directly. See a screenshot in Alexander von Studnitz post about Magento 2.
Objects (models, helpers, blocks, etc.) will be instantiated using this pattern: createObject(‘<class name>’) (e.g. createObject(‘Mage_Catalog_Model_Product’)) and not with Mage::getModel(‘catalog/product’). The method is of course still a factory which looks into the configuration for rewrites. Also the getTable-method will be changed: you will have to use getTable(‘catalog_product_entity’) instead of using the alias getTable(‘catalog/product’). The method will return the table name which might have a table prefix.
These changes have an impact on the compatibility of Magento 2 to Magento 1: Magento 1 extensions won’t work with Magento 2. It will be required to migrate extensions to be compatible with Magento 2. Magento will provide a documented migration path for that. I hope there will be a database migration tool, too.
Persistence
Magento 2 will provide support for the RDBMS MySQL, Oracle and MS-SQL.The refactoring of the resource model is already announced for the upcoming CE1.6 and EE1.11 release and will make following changes to the architecture of the persistence layer of Magento:
Refactoring of the resource model layer in Magento 1
The advantage of the refactoring is that the non-RDBMS-specific code is in the Resource Model and everything which is RDBMS-specific is removed and put into a helper for each supported RDBMS. This makes it easier to introduce support for additional RDBMS with nearly no code redundancy. The helpers are only needed when it is required to go beyond the limits of the database abstraction layer provided by Zend_Db for better performance or usage of additional functionality which other RDBMS do not support. Also the install and upgrade scripts will be abstract (upgrade-0.0.1-0.0.2 instead of mysql4-upgrade-0.0.1-0.0.2) and only when it is required, Magento 2 will still support mysql-*, oracle-*, etc. scripts for RDBMS-specific code. This refactoring will be backwards compatible.
A breaking news for me was that Magento is considering to remove EAV and only provide the EAV database schema for the catalog as an optional mode. This would improve the performance significantly, because there will be less JOINs in the queries and therefore less temporary tables which have to be created to sort and filter such results in MySQL. Also Magento would require less indexing which makes data imports and modifications of data faster.
EAV was build in since the start to support the modification of data models for products, categories, orders, quotes, customers and customer addresses without changes to the database schema. So basically it was possible to change the data model of these entities during the runtime of Magento in the Admin Panel. I was involved in a couple of medium- to big-size ecommerce projects on Magento in my career and I can say, that such kind of functionality is not needed and it also can be very dangerous for the stability of the Magento solution. Changes to the data model have to be part of a release and have to go through a proper release process which consists of QA activities such as testing the changes on a staging environment. This requires to perform such kind of changes by using upgrade scripts to ensure that the data model of all entities is the same on all systems (dev, acceptance, staging, production). And since the changes are part of a release and have to be deployed it is not a big deal that there are changes to the database schema.
Everybody who has watched the evolution of Magento 1 has also noticed, that EAV was removed step by step from Magento to gain better performance: first the quotes were stored in flat tables instead of the EAV schema, then the orders followed. Order items were flat right from the start (if I remember correctly) to be able to generate sales reports without too much pain. The flat catalog was introduced to remove the overhead which slowed down the rendering of category and product pages. The flat catalog is a special kind of an index, where each store has its own category and product table.
For me the logical next step is to remove the EAV for customer and customer addresses and last but not least from the catalog.
Web API
Although Magento has often announced to extend the API to cover more functionality, the API is still way behind the possibilities you have in the Admin Panel. Magento 2 will address this issue by including the extension of the API into the development process of new functionality. So all new features will be released together with new API calls covering the new functionality. Also the API will support JSON-RPC and REST in addition to SOAP and XML-RPC which are supported in Magento 1.
Maintenance
There are a couple of things which should improve the maintenance of running Magento sites.
The upgrade process will have a better workflow: the upgrade will be performed on a staging copy of the store (code and database). This staging environment can be used for tests before deploying the upgrade on the production environment. Magento solution partners should all do this already now with own deployment scripts or manually on staging environments. If not: shame on them!
The Magento profiler will output profiling entries in a hierarchy and not as a plain list like in Magento 1. This makes the profiler output easier to read. Also the output format can be configured to support Firebug and other browser plugins.
Wrap Up
At first I was expecting Magento 2 to go with the latest trends in the world of web applications. So called post-modern web applications (e.g. GMail) leave the classical request-response principle for loading complete HTML pages. They are designed like a native client-server application which has a powerful view running on the client and just fetches raw data from the server. I watched a nice presentation on the next09 by Malte Ubl in 2009, who was a Technical Director at SinnerSchrader at that time and now has joined Google in the end of 2010, with the title “Building post-modern Web Applications”. It covers this topic and you can watch it here: http://video.nextconf.eu/video/923175/building-post-modern-web.
Today, I think that the approach of Magento to make a major refactoring release is more pragmatic than changing the architecture of Magento completely. It is possible to release earlier and remove a lot of urgent issues outlined above faster to help the community to deliver better solutions for clients, better extensions for the community and better feedback for Magento. The benefits of this will form a better foundation for a big step to a modern web application design. The big question is how the X. Commerce strategy will influence the evolution and especially the revolution path of Magento. Will Magento be able to perform such big steps being a part of X. Commerce?
