Unable to provide the article you requested because it is currently pending approval for publication.

How we built Twittermail in 48 hours

By Lennaert Ekelmans (LennieZ)
We started Twittermail in june 2007, last week we launched our brand-new website Twittermail.com, this time we built it in CakePHP 1.2, in exactly 48 hours.
Okay, for those who don't know Twittermail, Twittermail is an openidea by Boris Veldhuijzen van Zanten, he thought that it would be nice if people could e-mail their tweets to Twitter.com.

This is very useful for people who use mobile phones. Of course you can go to the mobile webpage of Twitter, but sending an e-mail is much more easier. Some older phones do not even have a browser and only have e-mail functionality. Also a lot of businesses block Twitter.com on their corporate network but with Twittermail you can always update your Twitter account even from your business e-mail address.

Those who register at Twittermail, get a secret unique e-mail address like 1234abcde[AT] twittermail com. When you send an e-mail to this secret e-mail address, it will get posted instantly to Twitter.com through the Twitter API.


The Enviorement

Twittermail is a small project so basically we only need a sign-up form, a settings page and we need to show some recent activity. Since we handle loads of visitors and more than 3000 e-mails a day this project needed to be very scalable and thin. We read a lot about the benchmarking results1. We managed to do this project in exactly 48 hours, thanks to CakePHP (and Coca Cola).


Custom Validation

Users supply us their Twitter credentials so that we can create an unique e-mail address. Because we need to check if the information they supply is valid, we created a custom validation which checks the Twitter API whether or not everything is valid. In our model we created a custom function called 'isValidTwitter', in this function we make a CURL call to the Twitter API, we parse the output from JSON to an array and we get back the user information or an error.

Model Class:

Download code <?php 
class Twitter extends AppModel
{
    var 
$name 'Twitter';

    var 
$validate = array(
        
'twitter_username' => array(
            
'rule' => array('isValidTwitter'),
            
'message' => __('The credentials you supplied are not valid! Please try again')
        )
    );
?>

Model Class:

Download code <?php 
    
/**
     * Valid Checker: Is Valid Twitter Account
     *
     */
    
function isValidTwitter($value$params = array())
    {
        
$curl_twitter curl_init();
        
curl_setopt($curl_twitterCURLOPT_URL'http://twitter.com/account/verify_credentials.json');
        
curl_setopt($curl_twitterCURLOPT_HTTPAUTHCURLAUTH_BASIC);
        
curl_setopt($curl_twitterCURLOPT_HEADERfalse);
        
curl_setopt($curl_twitterCURLOPT_USERPWD$this->data[$this->name]['twitter_username'].":".$this->data[$this->name]['twitter_password']);
        
curl_setopt($curl_twitterCURLOPT_POSTtrue);
        
curl_setopt($curl_twitterCURLOPT_RETURNTRANSFERtrue);
        
$curl_result curl_exec($curl_twitter);
        
curl_close($curl_twitter);

        
$twitterResponse json_decode($curl_resulttrue);
?>

From now on you can do whatever you want with the results. In our project we encode the password, fetch some Twitter information and save it to the database.

Because we don't need a very complex User Authentication, we didn't use ACL.

Localization & Internationalization, l10n and i18n

At the moment our website is English-only, but we already made it multi-language proof by using the __('Login') function. For those who never worked with localization, in the cake console you can simply generate a language file by using the command 'cake I18n' here you get the option to extract all __('') strings to POI files.
You can read more about localization at the wonderful Cookbook at http://book.cakephp.org/view/162/Localizing-Your-Application

Scaling


Caching

Because we're serving more than 18,000 users and more than 800,000 e-mails have been sent, scaling is a important subject. First of all, use Cache! You don't need to get all data over and over again from the database. For example in the sidebar we have 'Newest Twittermailers'. This is cached for a year (yes, a year), everytime when someone is registering, we're resetting the cache. Cache isn't a thing you should forget, and for those who never worked with it please read the Cookbook.

Download code
// Reset / Delete the cache, so next time the newest Twittermailers list will be re-generated.
Cache::delete('sidebar_newesttwitters');


Containable Behavior

This is a very big improvement in the Cake 1.2 series, in the past I always used for my CakePHP projects 'expect' (For 1.1 users: http://bakery.cakephp.org/articles/view/keeping-bindmodel-and-unbindmodel-out-of-your-controllers).
This model behavior allows you to filter and limit model find operations. Using Containable will help you cut down on needless wear and tear on your database, increasing the speed and overall performance of your application. The class will also help you search and filter your data for your users in a clean and consistent way.
It helps you selecting only the data you really want, it's recommended to read the full recept about this at http://book.cakephp.org/view/474/Containable

Think about security

Often developers forget to think about security, but security is a very important subject when you're developing web-applications. Use the validations and double check them with the sanitize object built-in CakePHP.

Download code
App::import('Sanitize');
Put this code in your AppController and AppModel to achieve a higher lever of security, read about it at http://book.cakephp.org/view/153/Data-Sanitization/
For example:
Download code
echo Sanitize::html($untrustedString, true);

And remember: NEVER trust the input of your users!


Then there's the Model::save function, that lacks a good description in the cakebook, but in my opinion the $fieldList method is very important.
Like I said before, never trust the input of your users, supply $fieldList with an array of all the fields you're supposed to fill.

For example:

Download code
$this->Twitter->save($this->data, true, array('username', 'password', 'email'));


How we did it in 48 hours

Of course this is a (very) small project, but because of Cake we were able to manage this project in 48 hours. It's important to stick to the CakePHP 'rules', just use the validation methods, use the helpers, use the build-in components, it really helps you with keeping your code clean and compact.


Some respect

Show the world you're happy with CakePHP, put the 'cakephp - power' button on your website, be proud you're using CakePHP. Also don't forget to donate, the cake developers team are working day-and-night to supply us a better CakePHP, the only thing we could do is using CakePHP and show our satisfaction, at the end of 2009 we will donate the Cake Foundation 10% of Twittermail's profit.


Happy baking,

Bram Kok
Lennaert Ekelmans
http://www.twittermail.com

 

Comments 938

CakePHP Team Comments Author Comments
 

Comment

1 Awesome

Very cool.

How experienced were you guys with Cake prior to using it for this project? And how has it handled the traffic (did you have to install more hardware than before)?

Regards!
Posted Feb 26, 2009 by Giuliano Barberi
 

Comment

2 Trademark infringement?

Won't you be infringing twitters trademark by having a domain name and logo like that?
Posted Mar 9, 2009 by Thomas Pedersen
 

Comment

3 Nice!

Well done! Did you have any custom component to generate the secret email addresses for the users? It would be nice if you'd share them too. :)
Posted Mar 10, 2009 by Oliver John Tibi
 

Comment

4 Some replies

@Giuliano Barberi: We were kinda experienced, we built 3 others projects in CakePHP, but of course we have still much to learn.

Surprisingly we didn't had to add extra hardware, with CakePHP the serverload was incredible less than before. I don't really know why, probrably because of the intelligent queries and caching.

@Thomas Pedersen: We were one of the first Twitter applications were it was legal to have twitter in the domainname. So I guess it's no big problem, we like Twitter and Twitter likes us.

@Oliver John Tibi: Well Oliver, we just used a small snippet to generate a random code, here you see:


function _generateCode()
{
    $karakters = "abcdefghjkmnpqrstuvwxyz23456789";

    $randkey = "";
    $max=strlen($karakters)-1;
    for ($i=0;$i<6;$i++) {
      $randkey .= substr($karakters, rand(0, $max), 1);
    }

    // Check if the secret code already exists
    $findSecret = $this->find('first', array(
        'conditions' => array(
            'secret_twittermailcode' => $randkey),
        'fields' => array('id', 'secret')
        )
    );

    // Already exists?
    if($findSecret) {
        return _generateCode();
    } else {
        return $randkey;
    }
}

Posted Mar 27, 2009 by Lennaert Ekelmans
 

Comment

5 Checking for new messages in all mailboxes?

How do you guys go about checking for new messages in all your users' mailboxes? Is your script running every few minutes and looping through all email mailboxes that were created for your users? Or is there someway to notify your script that a new email is there waiting to be processed?

Thanks!
Posted Apr 12, 2009 by D
 

Comment

6 Smart

How do you guys go about checking for new messages in all your users' mailboxes? Is your script running every few minutes and looping through all email mailboxes that were created for your users? Or is there someway to notify your script that a new email is there waiting to be processed?

Thanks!

First of all, we just have 1 big mailbox (CATCH-ALL).

Then there's a pipeline which will push all new e-mails to a process script who's processing all the mails. This process is not in CakePHP but just plain PHP...

Thanks for your question :)
Posted Apr 12, 2009 by Lennaert Ekelmans
 

Comment

7 Is Twittermail Down?

I've been trying to access the twittermail site for two days and it looks flat busted. Any idea of when you guys will be back up? Thanks. W
Posted Jul 16, 2009 by will daly
 

Comment

8 Very Cool!

That's awesome ...

CakePHP is the best framework i ever worked on ...

over 20 BIG projects done by it in just 1 year
Posted Sep 14, 2009 by Moe Hassan