Once in a while I am asked by developers about components or frameworks that I recommend for general PHP Web application development purposes.
My responses often depend on the specific needs of each developer, but I think there are general recommendations that I could be useful to many of the PHPClasses site users.
Therefore I decided to write this post to share my thoughts on this subject so all of you reading this can reflect about the subject and withdraw your own conclusions.
* Free/Open Source software survey
First I would like to let you know about a request that I received from Dirk Jendroska of the department of Psychology of the the University of Wuerzburg in Germany.
Dirk asked me to invite the PHPClasses site users to participate in a survey on working conditions in Open-Source projects. So, if you can spare some time, please collaborate.
* Think for yourself before making your mind
Before proceeding with the frameworks subject, I would like to comment on recommendations and opinion making.
The Internet made it easy for people to communicate and exchange opinions very quickly. Sometimes you notice that people are talking a lot about a subject that may interest you.
All of the sudden, you have set yourself listening to every conversation that mentions certain trigger buzzwords. You may have noticed that the trigger buzzwords of the moment on the PHP community seem to be: frameworks, MVC, AJAX, Web 2.0, blogs, RubyOnRails, etc..
The point that I would like to make is that regardless of whatever are the buzzwords of the moment, do not get distracted, think about your needs and priorities before you let yourself be lead (or mislead) by whatever is the hot subject of the blogs or other sources of information you read.
In case you have not noticed, there are certain groups of blogs that are "working together" to form opinions and influence as many people as they can to move towards a certain direction.
Consider this detail very carefully, especially when you are reading blogs that seem to be personal, but in fact they were intentionally created manipulate people opinions. Often they have the as main intention to increase the sales of products of the companies to which the blog posters are affiliated.
There is nothing wrong on people influencing each other. What may be wrong is that you let yourself be influenced by opinions without reflecting or without considering first your needs and priorities.
Take in consideration that what follows is just my opinion. It takes in account my own criteria and conveniences. After you read this, you should reflect and realize what suites you well before you decide to follow any of the recommendations.
* What is a framework?
The Wikipedia definition seems to summarize it very well:
There are other definitions of frameworks in Wikipedia, but I think the one that applies to the context that has been discussed a lot recently in the PHP community is of Web application frameworks.
Basically, Web application frameworks are libraries of components and tools that are meant to minimize the effort to develop and maintain Web applications.
* Why there is a lot of people talking about frameworks now?
Web application frameworks are not a new subject. There always have been many Web applications frameworks for PHP since a long time ago. I remember for instance PHPlib, Horde, BinaryCloud, among others. So why all this "framework fever" now?
The impression that I got is that it is related to RubyOnRails. If you do not know it already, RubyOnRails is a Web application framework developed by David Heinemeier Hansson in a programming language named Ruby .
RubyOnRails was extracted from the development of Basecamp, a Web based collaboration site developed by a company named 37signals. David works for this company. This fact made me wonder if all this hype around RubyOnRails was somehow part of 37signals marketing plan.
Since I noticed a lot of people was talking about it, I became curious and tried to figure why this sudden RubyOnRails mania.
I went to their site, looked at the documentation, examples and some videos of presentations. I watched in particular this presentation video with almost 2 hour of length. It helped answering my question: "Why RubyOnRails?".
I learned that David is a former PHP developer that was somewhat unsatisfied with PHP 4. He claimed PHP 4 lacked of certain features that Ruby provides.
It was not clear to me which features he missed, but it seemed he meant features like exception based error handling and object class introspection (AKA Reflection). If my guess is right, a shift from PHP to Ruby would not really have been necessary now. All those features are available in PHP 5.
What is really intriguing is that RubyOnRails is often being advertised as an easier way to develop Web applications. However, if you watch the second half of that video, you can see David trying to demonstrate how to implement a pre-defined example.
At a certain point he proposes to write some code to implement a feature but then he chokes. He could not figure how to do it!?! If the author of the framework doesn't know, then who knows?
So, is RubyOnRails easier or not? Maybe David was just nervous and had a blank for a moment. Still I am not very convinced about all the fantastic slogans that you see on RubyOnRails site. It all seems greatly exaggerated.
Anyway, my point is that you take care when you switch from the certain to the doubtful. This applies not only to switching from PHP to another language, but also to switching between development methodologies or frameworks.
The truth is that there are no miracles. New methods and new tools require investing time and effort to learn about them.
If you are already using PHP for a while and you use a certain framework or methodology, switching to something else may be a step backwards. You may end up loosing time learning something that may make you less productive than before.
On the other hand, I noticed that RubyOnRails is very big in the Ruby community. That seems to be to the fact that there no other prominent framework for that language.
There is an old saying that states: "in the land of the blind, who has an eye is a king". I mean, RubyOnRails stands up in the Ruby community because it has no competitors. If it was PHPOnRails, I doubt it would get much attention because there is plenty of competition in the PHP world.
In any case, the point of this post was not to hit RubyOnRails. I am sure that David is very talented and RubyOnRails can be used to develop Web applications productively. At least David has the credit for putting Ruby on the map, which seems to be a big thing for Ruby, as it is not a new language.
Anyway, I am not encouraging or discouraging you to make an eventual switch to RubyOnRails. I am just advising to think about it thoroughly before you decide to make big moves. If you have the time to spare, it may be interesting to study how it works, and maybe learn something about its concepts, even if you do not plan on using it.
* What PHP frameworks are recommended?
This is one question for which I cannot give you a straight answer. It depends.
First you need to understand that there many frameworks. Some as specific for certain purposes, like AJAX frameworks for instance. Others are more broad, and involve many aspects of interest of Web application development.
* PHP 5 fate
One thing I noticed is that the latest bread of PHP frameworks only work with PHP 5. Despite this seems natural, the reality is that PHP 5 adoption seems to have been very small. Recently I mentioned this fact in a review of the Upgrading to PHP 5 book.
This review mentions a survey conducted by Nexen every month. Currently it shows that PHP 5 releases seem to only have been adopted in 6% of the public Internet hosts, while about 30% already use PHP 4.4, and 51% still use PHP 4.3 .
This must be very frustrating to all Open Source developers of frameworks and other packages that require PHP 5. One of the motivations to open the source of a project is to get as much feedback as possible. With such a small adoption of PHP 5, the feedback to PHP 5 framework developers will be very limited.
Zend has been working on their own framework. It is a PHP 5 only framework. Although, they did not seem to mention that, it suggests that their framework is yet another attempt to push PHP 5.
Pushing PHP 5 seems to be very important for Zend business. If the clients of Zend commercial products upgrade to PHP 5, they also need to upgrade to newer versions of those paid products. Therefore, the more Zend pushes newer PHP versions, the more money they can make. So, Zend has all the reasons to push PHP 5 as much as they can. Nothing wrong with that. Business as usual. I do not know if this is their main reason to push their framework, but it seems logic.
This is just one opinion, but I think that the main reason for developers and hosting companies not wanting to upgrade is related with the backwards incompatible changes that PHP 5 introduces. PHP 4 had already a bad track record of backwards incompatible releases. So, it is only natural that developers and hosting companies do not want to upgrade to PHP 5.
Anyway, I think that PHP 5 dependency is not the only reason for developers to not want to use a PHP 5 based framework. PHP 5.0 was released about 2 years ago. Therefore, most of these frameworks are necessarily younger. This implies that they are still in development or maturing. Relying on an immature framework is a risk that many developers do not want to take.
* Eat your own dog food
Another odd aspect about for Zend, is that they are not known for developing pure PHP projects for others. They developed the Zend engine in C and Zend Studio in Java. The Zend framework is for developing PHP applications. Which PHP applications have they developed in with Zend framework? None that I am aware of.
A developer that works on project for others but does not "try his own dog food" will always have difficulty to understand the limitations of his work and it will take more time to mature.
* PHP developers are not droids, they have feelings
Zend also claimed that they want to gather the best of breed packages in their framework including packages of different developers. This seems an utopia to me.
The problem is that Zend is only considering packages that only work on PHP 5. Therefore they are excluding all the other packages that have been developing in the previous 8 years of history before PHP 5 was released. Many of those packages are very mature and capable, without relying on PHP 5.
Besides this fact, there is also the resent that their initiative caused among certain active PHP community developers. I am not even talking about the developers that contribute to PHPClasses. I am talking about for instance about PEAR developers.
When Zend announced their framework, I noticed a lot of resent in the PEAR mailing lists. Some people seem to have felt there was some kind of betrayal.
PEAR is a project sanctioned by the PHP group, to which Zend developers belong. Zend has often promoted PEAR as a source of components for the same purposes of the packages they have included in Zend framework. I know that Zend used to spend money on Google AdWords advertising PEAR.
Despite Andi Gutmans of Zend tried to deny it, it certainly seemed like Zend framework is a step back away from PEAR. After all PEAR works under PHP 4, and so it does not fit in the apparent Zend PHP 5 agenda. That could justify reducing the interest on PEAR in favour of Zend framework.
This may be causing some bad propaganda to Zend framework because dissatisfied PEAR developers may end of discouraging everybody they can and influence to not use Zend framework.
Again, this is just my opinion, but all these factors compromise the adoption of the frameworks. A framework with lack of adoption may end up being canceled or stop being supported because it costs money to Zend and other companies that publish their frameworks for free.
* PHP lacks of formal standard specifications
I think it would all have been better for the PHP community if there was some cooperative effort, not to build more frameworks, but to build a common specification for a framework. From a single specification there could be one or more compatible implementations independently developed by different parties.
This is an idea that worked well with the Java community. Sun sat down with several prominent software development companies and produced many specifications for frameworks and platforms.
For some specifications they provided their own implementation for free. Other specifications have been implemented by different companies like IBM, Oracle and BEA. There are even Open Source implementations by the JBoss group.
Unfortunately the PHP community is often very disorganized and lacks of a truly cooperative vision. That is the reason why the community is so fragmented and he have too many frameworks, database abstraction layers, template engines, etc..
Personally I believe that cooperating is better than competing. However, this idea seems to be very hard to be agreed on the PHP community. With so many people competing to solve the same problems, each one in a different way, it is very hard to recommend anything.
This makes it hard for PHP to be accepted and taken seriously in the so called "enterprise world". I understand that Zend may be aiming to solve that problem with the Zend framework. I just do not agree that an unilateral development is the right way to go.
Anyway, if there is enough interest to move on and create effective standard specifications for PHP frameworks, count me in. It is about time that the PHP community starts having a real PHP Community Process that does not exclude anybody, similar to what exists for Java:
* So, what framework do I use? The NIH syndrome
Well, I do not use any Web application framework. I mean, I do not use a framework developed by others. It is not there are no good PHP frameworks out there.
- Do what I say but not what I do
The problem is that I suffer from the "NIH syndrome": Not Invented Here. This means that I develop 100% of the PHP code that I use. I do that because I it seems to be the most reliable way to develop code that works 100% exactly how I think it should be, without depending on the cooperation of others.
- Do not reinvent the wheel
Do not do this at home! ;-)
Just because I do it, it does not mean it is always a good idea.
Developing all my own code, means spending a lot of time developing, testing and debugging the code. I do it because I can. I work for myself. If I waste too much time on my own developments, it is my loss. If you develop all from scratch in your company time, it may be your company loss. Your boss may not like that at all.
- Do not break backwards compatibility
Besides the cost of developing everything from scratch, there is also the problem of spending time researching the best solutions for each problem you need to solve.
If you start developing something that others already developed before, chances are that your solution is less mature and you may need to rethink it later. This is bad for Open Source libraries because it may lead to introducing backwards incompatible releases.
If you are only developing for yourself, that is fine, but if your are opening the source of your projects, breaking backwards compatibility deeply annoys your community. Remember my comments above regarding why developers and hosts are not upgrading to PHP 5 .
Anyway, developing your own framework, class library or other kind of development tools is not bad. For many, it may just not be a viable option.
- Wrap and reuse
Over time I have seen several projects that reuse packages that I wrote. However, some people did not like certain details of the way my packages are provided.
Some did not like the coding style, others naming conventions, others just want to use my package as a temporary solution until they can develop an equivalent solution that matches better their needs.
Teoni Valois is a contributor of the PHP Classes site. He was also nominated to PHP Programming Innovation Award with a package to generate PHP classes from UML class diagrams saved in OMG XMI format.
Some time ago, in a PHP experts mailing list, in a discussion precisely about the frameworks topic, Teoni explained that he solved this problem of using existing packages as a temporary solution by creating sub-classes of the packages that we wanted to reuse.
The sub-classes implement the API he wants with the style he wants. Later if he decides to implement the classes by himself, he can keep the API that he defined, and so he does not need to change his applications.
BinaryCloud is long standing PHP framework. I recall that they reused several classes of mine with an approach similar to Teoni's. This allowed them to switch the use of some of the packages to others that they preferred.
- Minimize dependencies to reduce memory usage and increase scalability
I have developed a collection of packages over the last 7 years, since 1999 when OOP was introduced in PHP 3. This collection consists mostly of general purpose classes that can work independently.
A reduced number of class dependencies is important, so you can minimize the amount of code that each page loads when it is accessed.
If you use a PHP caching extension, like for instance Turck MMCache, you can see in the statistics page that in average for each 1Kb of PHP code, it takes 8Kb of memory to load and execute per Web server process. If you include too many needless classes in memory, obviously that can severely limit your Web server scalability.
From what I could understand, dependency minimization has also been a concern with the packages included in the Zend framework and other some other PHP frameworks.
- Provide examples and documentation to encourage adoption
I have some packages that were not yet published for the lack of time to produce sufficient examples and documentation.
Examples and documentation are very important to encourage others to use your work. That is why these aspects are considered separately for the user ratings of each package published in the PHPClasses site.
- My package selection
Since I only use my own class libraries and tools, it may now be clear for you why most of the time I only recommend my classes and not others. It is not that others do not deserve to be mentioned. I think that the people that actually use other frameworks or class libraries are more qualified comment about them.
I am going to give an overview of what I use to develop the PHPClasses site and other sites that I have. Feel free to comment on this post to tell about how you implement your own framework and what advantages you see in your approach.
Most of my packages have already been published in the PHPClasses site and are listed here:
Let me just mention the ones I consider most important and I reuse more often.
- Database abstraction: Metabase
The PHPClasses site always used MySQL. Despite this fact, since the beginning there was a concern that someday I would need to switch to a different database. That was the main reason why I developed Metabase.
This is a database abstraction package that allows you to write RDBMS
independent applications without having to change a single line of application code if you want to work with a different database type.
Metabase portability features go way beyond database access independence. Metabase also allows you to define and install you database schema in a XML file format independent of the RDBMS.
When you want to change the installed schema, you just need to edit the schema XML file and ask Metabase schema manager to apply the changes. Metabase upgrades the schema without loosing the information you stored in the database tables since you installed the database for the first time or upgraded from the last time.
This schema management features has been a blessing. In the development environment I update the database schema with as many changes as necessary to implement new features. When it is ready to install the new feature on the server, I just update the files on the server and ask Metabase to alter the schema safely, regardless of the version of the schema that was installed on the server before.
Metabase has been inspiring other PHP database abstraction packages since it was release for the first time in early 2000 . Metabase code is also the base of PEAR::MDB and the follow-up PEAR::MDB2 projects lead by Lukas Smith. Metabase and PEAR::MDB2 are almost the same project just maintained by different people. PEAR::MDB2 satisfy the PEAR coding rules.
- Content caching: File cache class
One thing that I learned from dealing with the never-ending growth of the audience of the PHPClasses site, is that the fastest database applications are those that avoid accessing databases as much as they can.
For content Web sites, the database accesses are the greatest bottleneck. The less you access the database, the faster the site becomes.
One way to avoid accessing the database, is to cache frequently accessed content. I am not talking about just the database query result sets, but rather the actual HTML pages generated from the query result data.
For that I use this cache management class that stores the cached pages in server side files. It is very robust as it prevents that simultaneous attempts to update a cache file corrupt the data.
Currently the PHPClasses site is using 225,000 cache files that occupy over 1.3GB of disk space.
Here you may find more details about database applications performance smart caching and other tips:
- Forms: Forms generation and validation
Forms handling are an essential component of Web sites. This class is the one that motivated the creation of the PHPClasses site. It is by far the most popular class I ever wrote.
It has an plug-in based infrastructure to let the developers write enhanced form components. Several plug-ins are available like the CAPTCHA validation, calendar date chooser, linked select inputs and a special plug-in for submitting forms without page reloading using AJAX.
- Site searching and mirroring: Htdig site indexing and searching interface
Nowadays the site searching is carried by Google. It is both a means to offload the site server and generate additional advertising revenue.
There is also an internal search engine based on the Open Source package Ht:/Dig . This package calls the htdig commands to crawl the whole site at least once a day and keep the search index updated in a much faster way than Google could achieve remotely.
This internal search engine will play an essential role in the upcoming premium services for paying subscribers. More on this later.
Another non-obvious role of this package is to create an internal mirror of the site. This internal mirror extracts the static version of the site pages that is replicated to the site mirrors using rsync.
Each mirror gets updated pages at least once a day. Currently a mirror contains over 21,000 pages with a total of 325MB . Thanks to the technological miracles of rsync, it is possible to keep updated about 50 active mirrors world wide, with reduced load of the main server and minimized bandwidth consumption.
- Web services: HTTP protocol client
There are certain resources that the site needs to fetch remotely like RSS feeds and other resources that are available to authenticated users of certain remote sites.
To get remote RSS feeds a simple PHP file_get_contents call would be sufficient.
However, the site needs also to automate the integration of remote sites like Yahoo! groups. It is necessary to send invitations to certain support mailing lists, like the mirror hosts mailing lists or the premium beta tester users mailing list.
Since Yahoo Groups does not yet provide a Web services API, sending the invitations to join requires to authenticate with Yahoo and submit the invitation form.
This class helped solving that problem because it can emulate a browser submitting forms, collecting cookies, handling redirection and whatever is needed to automate the access to remote sites like a normal user.
- Logging: Log watcher
There is nothing special about logging the PHPClasses site activity. In some cases it users simple file updating function.
In more serious cases the site uses the PHP error_log and trigger_error functions. Fatal errors cannot even be trapped with PHP code. In these cases it is important to receive a notification because serious errors need to be fixed promptly.
The solution that is employed is a log file watching class. It is run in the background, started as a cron task. It monitors the PHP error log file periodically. If the error log file contains newer lines added after the last update, it sends an urgent e-mail to my e-mail address with the newly added lines.
I cannot tell you how many times this solution saved long down time hours, some times caused by human mistakes, other times caused by exceptional use of server resources.
- Email sending: MIME message composing and sending
The PHPClasses site sends a lot of e-mail messages. It is sending about 5 million newsletters and new content alert messages every month.
It is very important that these messages are composed in standards compliant formats, so the site newsletters are not confused with spam.
This class makes it easy to compose and send all sorts of standard compliant e-mail messages in text, HTML, with attachements, with embedded images, etc...
It also provides workarounds to several very frustrating problems that PHP users have when using the PHP mail() function, like incorrect formating and the need to authenticate with a SMTP server.
It also provide means to optimize deliveries to many users. This helped saving a lot of resources when sending messages to many subscribers, like fo instance the site newsletters.
One less common purpose that can be achieved by this class is the ability to send very urgent messages. Some people believe that flagging the message as urgent will make the message be sent faster. Usually that flag does not make a difference.
What can make a difference is to inject the message directly in the destination SMTP server. Urgent delivery mode is provided by the SMTP class message. The MIME message class should be used in conjunction, so it can format the message properly before sending.
- Email receiving: POP3 class
Certain automated procedures require that the site deals with incoming e-mail messages, like newsletter unsubscribe requests or mailing list bounced messages.
The POP3 class is used to fetch the incoming message from mailboxes, specifically to receive automated requests sent by e-mail.
- Syndication: RSS writer
RSS is an important means to let others know about new site content.
This class provides means to generate standards compliant RSS feeds, with support for defining special tags using namespaces for RSS modules for specific purposes.
One innovating use of this class is to serve the site search result listings in RSS format. This has been used by Phuse, an experimental search engine that unifies the search results of several sites of the PHP Developers Network.
- Site configuration
The site configuration is defined by a custom class with variables that define settings values to be used by the site. These settings may be different in the production and in the development environment.
I have seen other developers using .ini files for configuration. I have preferred to use a simple configuration class because it can be automatically cached by a PHP caching extension, like Turck MMCache. This ends up being much faster than parsing .ini files on every page script execution.
Additionally, I use a separate script that is loaded from the site configuration class to override default values. This way I can customize different settings that are different between the production and the development environments.
When you want to develop a site that may appear in different idioms, you need to provide means to display all the texts in the current user preferred idiom.
The PHPClasses site was planned from the beginning to support multiple idioms, despite it currently only appears in English.
The solution that is used is to simply have separate PHP scripts that define all the strings for each of the site sections. These scripts are loaded in a separate directory for each supported idiom.
I also considered using gettext, but I wonder if making gettext read locale files would be faster than loading PHP scripts. Like the configuration classes, if you use a PHP caching extension, loading text string scripts may be faster to load than using the gettext extension.
- Object Relational Mapping: Metastorage
I the beginning of the site I used to develop DAO (Data Access Objects) classes that performed most of the operations to store and retrieve in the database the objects with information about site entities, like the user accounts, session information, packages, files, etc...
Those DAO classes allowed me to separate the access to the site information stored in the database from the site business logic code. That was all fine but it used to consume me a lot of time to write, test and debug such classes by hand. It required writing a lot of SQL manually.
In 2002 I started investing a lot of time and effort in a code generation tool named Metastorage. It is capable of generating these DAO classes from a high level definition of the site entity objects model.
That model definition can include the classes with the persistent variables, validation rules, relationships between classes and functions to execute the different type of operations to manipulate those objects.
Metastorage can generate all classes of the persistent objects with all the code to execute the necessary database queries to perform what is usually know as Object-Relational mapping.
That is not all. Metastorage is also capable of generating a database schema definition in the Metabase format and a special class to install it.
Metastorage is now very mature. Besides typical Object-relational mapping code generation, it can also generate report data extraction classes. These classes are ideal to extract information of related objects of many different classes, all using a single database query.
This is recommended for extracting data for read only operations, i.e. operations on large amounts of data that do not alter the database contents, like for instance generate report listings or sending newsletters to many users.
Persistent object classes are not good for this purpose. To extract the same information from multiple classes, they usually require executing multiple queries, as they only access one class per query.
Using persistent object classes for this purpose is a common mistake of developers that use the ActiveRecord design pattern for bulk data fetching.
Another good point about Metastorage is that it generates self-contained code. This means that the generated code does not require any run-time persistence library to access the data objects, besides the database abstraction package.
Metastorage also supports an Object Query Language (OQL). It is very similar to SQL but has an object oriented syntax. The OQL lets the developer express complex object search conditions that can be used to filter the list of retrieved objects to just a subset of interest. The OQL expressions are compiled into optimized SQL at code generation time.
All these aspects make it possible for Metastorage generate very compact classes that execute much faster. The reason for this is that the phylosophy of Metastorage is to anticipate slow optimization decisions at compile time, instead of wasting time and performance on evaluating these decisions at runtime.
Metastorage has already been used to implement several systems of the PHPClasses site like the forums system and the blog system. If it was not for Metastorage, these systems would probably be still in development. The time that I took developing Metastorage was certainly well spent.
- MVC: Use case mapping
I do not use any framework to implement Model-View-Controller (MVC) separation. As I mentioned in past posts, I use a methodology name Use Case Mapping.
This is a methodology that I follow since a log time ago. I use it to implement each application use case using a single class.
Use cases are the situations that each application implements, like for instance the login page, the subscription page, the blog posting page, and so on.
The MVC aspects are separated in different functions of the use case classes. Actually, the model aspects are implemented by separate classes generated by Metastorage. The controller and view aspects are implemented by different functions of the same use case class.
Traditional MVC implementations use separate classes to implement each of the model, view and controller aspects. I consider this approach needelessly more complex. The problem is that when you need to share information between each of the classes, you need to pass it by argument when you calling each class.
With the Use case mapping class approach, common information to be retrieved from the model classes or shared between the controller and view functions, is stored in private variables of the use case class. This way I avoid the complication of passing information back and forth by parameter.
Well, believe it or not, this is the end of this very long post. If you read it all from the beginning, thank you for your time and patience. I hope it was worthy.
As usual, if you would like to make corrections, or your have questions or criticisms to make, feel free to post your comments.