It’s not what you look at that matters, it’s what you see. — H.D. Thoreau

Most performance conversations, at one point or another, will get to the subject of caching. And for a good reason. Caching helps to hide some of the biggest performance inefficiency in many systems. That’s right, hide not fix. Whether it’s local storage of data, pre-calculated data set or a transient storage to reduce a number of server-side requests, all these methods are improving performance by limiting the frequency of expensive operations. Again, it is important to remember, cache does not improve the performance of the operations themselves, but rather reduces/delays/eliminates the need to execute those operations.

Don’t get me wrong, limiting a number of expensive connections or operations is important, this is why caching is a common component in most modern architectures. This is why it’s somewhat surprising to me that other good practice techniques that accomplish similar goals, but don’t fall under traditional definition of “caching”, often get ignored. Specifically, I am talking about scope caching™.

What is “scope caching”™?

Scope caching™ is something that most of you familiar with programming are likely already doing in some capacity. The simplest example of scope caching is a basic programming principle of not putting computational operations in a loop where possible. Running operations outside the loop, storing it in the variable and using a result variable in the following loop/code path is an example of reuse of pre-computed value. Which, in essence, what caching is trying to accomplish.

How many scopes are there?

Many. As many scopes as there are in a programming language and than some. That’s why it’s important to take time and understand the needed scope of cache requirements.

For example, scoping a variable globally is considered a bad practice. However, when working with a web language, global scope could be a webserver thread or webserver session. Loading something once and keeping it for the life of that scope for all the users can (and should) be completely acceptable solution. Pre-generated constants is a good example. If you (like many) store your countries and states in the database, loading them during a web server start and keeping them in global memory, instead of going to the database on every request, is a form of scope caching.

And speaking of web, REQUEST and SESSION are probably the two most common scopes that can be taken advantage of.

Consider a function that sits on top of auth service, that returns a current user object. It’s really simple. Useful. Widely used.

public function currentUser() {
  var currentUser = classes.getCurrentUser();
  return currentUser;
}

A lot of different modules and services take advantage of it to validate user’s access control, setting, etc. And because of that, it’s being called, at minimum, 14 times per page load. 14 requests to your database/cache/user service, over the network, to retrieve the exact same information on a single page. Service design ftw.

Problem? Of course. But an easily solvable problem, without the need to re-engineer the service.

public function currentUser() {
  if ( !isDefined("REQUEST.currentUser") ) {
    REQUEST.currentUser = classes.getCurrentUser();
  }
  return REQUEST.currentUser;
}

Storing a result of the first-retrieved user object in a REQUEST variable, persistent for the duration of that, well, request, will make the information available to different consumers on the same page without a round trip to the source.

Scope caching™!

The list of scopes above is not exhaustive, although it (hopefully) provides a good illustration of the diversity of scope levels and techniques to limit operations. Remember, if you need to ask for the information more than once, make sure to cache the response. (hey, it rhymed!)

Caching: Understating the scope was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Unless you are running a complete brochureware site, chance are, you have some sort of content personalization requirement, whether it’s user’s name, number of items in the shopping cart, custom navigation or something entirely different, specific to a particular user.

Let’s examine a prototype of a standard e-commerce personalization logic.

When a user logs in, on success, we store their identifier in a session variable (commonly — browser cookie). When that user navigates to a different page, we retrieve user information (to acquire first name to display) based on the identifier stored in a local cookie and then, if the user exists, we pull user’s shopping cart information to display a number and, in this case, amount of items in the cart.

From a code perspective, the logic is pretty simple. You add a few line snippet of code to your header and on every page you now display information personal to that particular user. Easy.

var loggedUser := user.New(cookie)

if len(loggedUser.firstName) > 0 {

  loggedUser.fillShoppingCart()

}

But the ease of implementation comes at a cost of a performance hit. First of, we just added two new lookups to every page. Furthermore, because every page is now a page that has dynamic content, pages cannot be easily cached. Which, in turn, means that every page you serve will always need to perform multiple lookups to retrieve the information needed to be custom for that individual user. Which sucks.

Note, the lookup implementation logic for both user and shopping cart information can vary. Most commonly, those calls query the database for the data. Information can also be retrieved from APIs (both internal and third-party), which, given JavaScript AJAX lookup ability, can be done completely in the front end code, partially solving “no caching” issue. However, API lookups come with a set of additional performance challenges that need to be considered.

Luckily, in a lot of cases, those lookups can be minimized, if not eliminated altogether.

Store all the information you need in cookies.

That’s it. For the example above, in addition to storing user id in cookies, also store user’s first name. And update cart counter (and maybe an amount) cookie every time the user updates the cart. This way, instead of doing server lookups, the information will be available directly in user’s browser.

There, of course, may be a reasonable concern about data tempering. Since browser cookies can be altered by a user, a concern would be that shopping cart data can be manipulated to rig the checkout process. An important thing to remember, the actual shopping cart page doesn’t have to (nor is recommended to) rely on the cookie for item information. This solution is intended to reduce the number of request calls that would’ve been made by every page. A database or an API lookup on a single page is not a problem we’re solving. Or should really be trying to solve in this case. Checkout process is one of the areas where a small performance hit is worth the data immutability guarantee.

Local Personalizations was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this day and age, more and more applications leverage external services. So much so, that only about 6% of all sites has no 3rd party content and almost 40% of all site have between 75% and 99% 3rd party content. And for a good reason. External services provide a tremendous benefit, not only in brand familiarity and content, but also in cost saving, considering you’re leveraging years of someone else’s work for your gain.

However, one of the major risks when implementing a third party service is the reliance on the availability of that third party service. Availability of your production site. Based on the service that you have no control over. And no matter how large or successful the service that you’re using is — it will go down (or become unavailable) at one point or another. Perhaps the best example of it is using Google CDN for jQuery, only to realize that your site will break for all Chinese users, since, you know, China blocks all traffic coming from Google.

And so we have a conundrum. We should (and do) use 3rd party services, sometimes extensively, yet our systems become more volatile because of that. Luckily there are a few golden rules to follow when implementing 3rd party service.

1. Only connect to a third-party service where needed. Don’t try to connect to SSO on every page load to validate that a user is still, in fact, the same user you displayed the previous page to. Cache the results locally.
2. Avoid external connections on critical path. This is the core idea behind putting all your JavaScript in a footer of a webpage. Don’t load Google Analytics as a first thing on your page, you will delay the display of the content that actually matters. Make the connections after your content is loaded, or better yet — connect asynchronously.
3. Trap time-outs and errors. You do it with your database connections, why would you treat external connections differently?
4. Create a fallback plan. You have no control over external services, but you do have the control over the content presented to your users. If external images are the essential feature of your site — store them locally after the pull, so you can fall back to the latest available content in case the service is unavailable. Remember, often times stale content is better then no content at all.

So how does it all translate to a real example? Let’s take a basic example of pulling data from external source in JavaScript with jQuery XHR call.

$(function () {
  $(document).bind("ajaxComplete", function (e, xhr) {
      if (xhr.statusText == 'timeout') {
        // pull local content
      } else {
        // use content from response
      }
  });
        
  $.ajax({
    url: 'http://example.com/feed.json',
    dataType: 'json',
    success: function (data, status, xhr) {
      // generate consumable content from response
    },
    error: function (xhr, status, err) {
      // pull local content
    },
    timeout: 3
  });
});

XHR requests are asynchronous by nature, which automatically removes the load time from critical path, allowing your content to be loaded independent of 3rd party service latency. Error trapping also accounts for blatant service unavailability or problem by pulling local content if error is detected. And a part that most people forget, latency detection is also enforced via timeout flag and check of response status.

That’s all. A couple of parting thoughts:

• In modern web applications external dependencies can be nested. Use the sanity check list above for all of them.
• Latency is a performance killer. Make sure you utilize caching to reduce number of external requests to an absolute minimum.

External dependencies was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Every type of content was not created equal. Nor should it be served the same way. At a high level, you can separate your content into 3 groups:

• infinitely static — image is a perfect example of this. Once an image has been published, you should never update it (because multiple reasons) and should be able to cache it almost indefinitely.
• temporarily static — this covers the content that renders into static output. CSS, articles, etc. The content that may change in the future (requiring you to purge cache) but generally can be cached on demand for extended period of time.
• dynamic — the content that is individual for each user/load. Personalization, user profile/history, top/recent content — all these cannot be cached for any extended period of time and require re-computation or rendering on each request.

Not only those groups should be treated differently from cache strategy perspective, but they should also be treated differently when serving them from origin.

The overhead

This applies to any web server (nginx, node.js, Apache), but for the ease of example, let’s use Apache. Chances are, your httpd.conf starts with something like this (with correction to language used).

# load modules
LoadModule perl_module /opt/apache22/libexec/amd64/mod_php.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_apreq2.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_ssl.so

# serve on port 80 for non-ssl content
Listen 0.0.0.0:80

# setup mod_rewrite engine
RewriteEngine On
RewriteLogLevel 4
RewriteLog /www/logs/apache/rewrites.log

# validate against rewrite checks
Include /www/etc/httpd-rewrite-global.conf

# define server/thread limits
StartServers 5
ServerLimit 40
MaxClients 40
MinSpareServers 5
MaxSpareServers 10

# if the file doesn't exist is static directory 
# process dynamic content through the dispatcher function
RewriteCond /www/htdocs/static/$1 !-f
RewriteRule /(.*)$ /www/htdocs/dynamic/dispatcher.php [L]

# if file is static and is found - serve it
RewriteRule /(.*)$ /static/$1 [L]

Most people don’t think about it, but with this setup, every request is going to load (or use) required modules and try to compare the requested URL to a (usually) long list of rewrite rules (that have a tendency to accumulate rather quickly). So the question you have to ask yourself is — does serving logo.png require loading mod_php module and does it need to be matched against dozens of legacy URL strings the business decided to keep active after the last redesign? Or, in fewer words, does every static asset need the same overhead as dynamic content? The answer is — no, no it does not.

Furthermore, is the service/thread configuration you use for your dynamic content is the same as you would need to serve your assets from origin? And again, the answer is likely no.

Separation of responsibility

Separating the serving of different types of content allows you to optimize and tune serve time for each individual type of content. The way to accomplish it is to have individual web servers running for each type of content that requires individual optimization. For example, instead of having a generic httpd.conf that listens to requests on port 80, you need to create httpd-static.conf listening on port 80 and httpd-dynamic.conf listening on different port (let’s say 8081).

httpd-static.conf

Listen 0.0.0.0:80

# static content configuration parameters
StartServers 2
ServerLimit 7
ThreadLimit 1200
ThreadsPerChild 200
MaxClients 1200
MaxRequestsPerChild 0
MaxSpareThreads 1200

RewriteEngine On

# If it’s a local static file, go ahead and serve it, 
# otherwise send to dynamic web server
RewriteCond /www/htdocs/static/%{REQUEST_URI} !-f
RewriteRule ^(.*)$ http://localhost:8081/$1 [P,L]

httpd-dynamic.conf

# load modules
LoadModule perl_module /opt/apache22/libexec/amd64/mod_php.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_apreq2.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_ssl.so

# serve on port 8081 for non-ssl dynamic content
Listen 0.0.0.0:8081

# setup mod_rewrite engine
RewriteEngine On
RewriteLogLevel 4
RewriteLog /www/logs/apache/rewrites.log

# validate against rewrite checks
Include /www/etc/httpd-rewrite-global.conf

# define server/thread limits
StartServers 5
ServerLimit 40
MaxClients 40
MinSpareServers 5
MaxSpareServers 10

# serve content through dispatcher
RewriteRule /(.*)$ /www/htdocs/dynamic/dispatcher.php [L]

Use httpd-static.conf as a gateway, to quickly serve static content and pass on the dynamic content to httpd-dynamic.conf to do the processing.

You can configure static content to also honor individual extensions instead of files from certain directory. And in addition to having separate configuration parameters, tuned to individual types of content, this allows to easily add different logging/monitoring/caching rules to different types of content.

Content separation was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Loops are generally not great from the performance perspective, but often times it’s relatively easy to pinpoint and optimize those inefficient function you wrote and call inside of a loop. But sometimes, especially with JavaScript, the functions causing the performance hit are the ones that are provided to you by language.

Let’s look at the most basic example of taking an array of items and adding them to your html table.

for (var i = 0; i < items.length; i++) {
  document.getElementById("listGrid").innerHTML += "<tr><td>" + 
                                                   items[i].name + 
                                                   "</td></tr>";
}

This is pretty straight forward. Iterate through the array and inject each item name into a listGrid table . There are no special functions to worry about, this loop is based purely on JavaScript-provided functions for doing exactly those things. And yet, there is still room for optimization.

Reduce property lookup overhead

First of all, let’s look at the loop definition itself. The instruction is to iterate from index 0 to last element of the loop, as defined by the array function length. Problem here is that items.length property is being accessed on every iteration of the loop, for really no reason. So let’s take it outside the loop.

var itemArrLen = items.length;

for (var i = 0; i < itemArrLen; i++) {
  document.getElementById("listGrid").innerHTML += "<tr><td>" + 
                                                   items[i].name + 
                                                   "</td></tr>";
}

Now, no matter how many elements are in that array, items.length will only be looked up once.

Reduce DOM access

The other issue with the example code is the injection action itself. document.getElementById traverses DOM with each loop iteration to find the same listGrid element every time. Instead, let’s just do the lookup once, store the element in a variable and use it in a loop for injection.

var itemArrLen = items.length;
var element = document.getElementById("listGrid");

for (var i = 0; i < itemArrLen; i++) {
  element.innerHTML += "<tr><td>" + items[i].name + "</td></tr>";

}

Note, the same applies if you’re using jQuery (or other frameworks) instead of raw HTML for DOM manipulation. Remember, no matter how you to do DOM lookups — you’re still doing DOM lookups.

Reducing property access was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Perhaps the most effective caching mechanism is in-memory cache. Tools like memcached and redis give a great out-of-the-box capability to store and retrieve your content to/from memory and effectively manage TTL. But what can you do when you don’t have those tools available?

Never fear, you can implement your own on-disk cache. While not as effective as in-memory cache, it would still provide a significant relief to system under heavy load.

The basics

The premise is simple. Take any function that retrieves, generates and returns content from any source (such as database queries, API calls and third-party content providers) and add a conditional to it.

sub generateContent() {
  if (has_cache($cache_file, $cache_ttl)) {
    return _read_cache($cache_file);
  } else {
    return _write_cache($cache_file);
  }
}

The conditional will check if the cache file exists and if it does, get content from cached file instead. If cached content doesn’t exist, generate content and store it in cache. Simple, right?

_read_cache and _write_cache functions are as simple as you imagine them to be. It’s a basic file manipulation: open, read/write, close.

###### FS caching

sub _read_cache {
    my $cache_file = shift;
    open (my $fh, '<', $cache_file) 
      or die "Can't open file doe reading $!";
    read $fh, my $cache_content, -s $fh;
    close $fh;
    return $cache_content;
}

sub _write_cache {
    my $cache_file = shift;
    my $content = generateContent();

    open(my $fh, '>', $cache_file)  
        or die "Could not open file '$cache_file' $!";
    print $fh $content;
    close $fh;
    return $cache_content;
}

The only remaining missing piece is cache validation. Unfortunately, because you’re building a cache system from scratch, unlike in-memory tools that manage your TTL for you, you need to implement your own TTL management. You can simply check the last updated timestamp of the cache file against your desired cached time. Done!

sub has_cache {
    my $cache_file = shift;
    my $cache_ttl = shift; #in seconds

    if (-f $cache_file) {
        # in seconds
        my $last_updated = (stat($cache_file))[9];
        my $now = int (gettimeofday);
        my $diff = $now - $last_updated;
        return 1 if ($now - $last_updated <= $cache_ttl);
    }
    return 0;
}

The caveat

There are only two hard things in Computer Science: cache invalidation and naming things — Phil Karlton

The advantage of this cache is that it’s persistent, even if you reboot the machine it’s on, the cache files will still be available. The drawback — it’s locally disk-bound. Which, among other things associated with disk bound processes, means you need to have a separate cache on each of the nodes. Which, in turn, means that you need to make sure you invalidate the cache across every node on-change. There are numerous ways of doing it, like setting up listeners on each node or implementing timed or on-demand cache purge instead of on-change triggers. Whichever way you decide to implement it, be mindful of the problem you’re facing. Otherwise you’ll be left with a problem worse than stale content — inconsistent content.

Caching: File system was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

The easiest way to ensure that your web requests are going to be cached by front end consumers, such as browsers, mobile devices and proxies, is to defined Cache-Control header for all the resources that you serve.

At the most basic level, all you have to do is add one line to your .htaccess file (if you’re using Apache):

# cache all resources for one month
Header set Cache-Control “max-age=2628000, public”

This will direct front end consumers to, much like the comment says, cache all the pages and files you serve for a month. So if a user hits your webpage and returns to it tomorrow, that request will not hit your server, but rather be served from the browser cache.

That’s great and all, but chances are, your site is serving dynamic content that changes a little more frequently than once a month. So perhaps you want to put some logic around the caching directives to differentiate the content and TTL for different types of files.

# cache static assets for one month
<filesMatch “.(jpg|jpeg|png|gif|js|ico)$”>
  Header set Cache-Control “max-age=2628000, public”
</filesMatch>

In this example, the directive is to cache only images for one months. Other types of files will be served without cache directives.

You can add more complex rules, such as setting cache directives for individual files and determining TTL on them based on the volatility of the content served.

You can also define header directly inside your application pages, to add even more granularity and flexibility for offloading content serving on to the consumers. For example, if you’re using PHP, you can add a header directly into an individual page:

# cache this page for one minute
<? header(“Cache-Control: max-age=60”); ?>

And for those thinking that one minute cache is not going to help much, think again!

Caching: Cache-Control header was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Rules of web application performance

Performance != Scalability

Those two concepts are not two of the same, even though they are often used in conjunction when talking about web application performance. Best way to think about the difference is to think about the fast food franchises. Scalability is building more restaurants in the neighborhood to serve more people that are moving into the neighborhood. Performance is adding automated ordering system to streamline the process and serve more people in a single location. This is why cloud solves the horizontal scalability problem, but does not, in any way, solve the performance problem.

Performance is technology agnostic

It doesn’t matter if you’re a Ruby person or prefer Python. Whether you’re still using proven Apache or prefer to keep your code stack in JavaScript and using Node.js. Similarly, if the only database you’re familiar with is MySQL or you’ve had your hands on PostgreSQL, Oracle and even, dare I say, DB2. It doesn’t matter what the technology of choice is, the core principals and techniques to improve the performance are the same across the board. Actual implementation details (config tweaks, syntax, etc.) may slightly vary depending on technology, but the concepts stay true.

Performance is not a feature

Much like with security, you cannot add “performance” as a line item to your project plan. Performance impact should be considered at every step of design, architecture and development. Because of the cascading nature of performance problems, early omission of performance considerations can cause major outages. Remember, not all the performance problems can even be tested for in development/staging environment. Some can only be observed in production.

Performance is subjective

That said, don’t over engineer for the results you don’t need. Keep in mind the effort vs reward trade off. Optimize for what’s needed and what’s acceptable. And remember:

Premature optimization is the root of all evil

— Donald Knuth

Performance mantras was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Migration scrips. We’ve all had to write them. They are the background, throwaway scripts that don’t have any performance requirements. Who cares if the script you planned to run overnight takes an extra hour to finish, especially since it doesn’t directly impact users? And generally all of that is true. Of course, when the situation falls out of the “generally” bucket, it fails spectacularly.

$max_size = 4200479;

for ($i = 0; $i <= $max_size; $i++) {
  // connect to the source database and process row
  $dbh = new DB(…);
  $sth = $dbh->prepare("select * from source_table where id = ?”);
  $sth->execute($i);

  _process_row($sth->fetch());

  // connect to the target database and store row
  $dbh_trgt = new DB(…);
  $sth_trgt = $dbh_trgt->prepare(insert into target_table ... ”);
  $sth_trgt->execute(...);
}

This is a pretty common base migration script. It iterates through the rows in the old table (4,200,479 of them), modifies the data and structure to fit the new schema and inserts the row into the new table. Now, it has a lot of performance issues (n+1 probably being one of the most visible one). But do we care? This is a one time script that will never be used after the migration is complete. Right?

As it turns out — we do care. The script above would take about 6 seconds per row. Which seems decent enough for a throw away. So, to complete the full migration it would take 25,202,874 seconds. Or a little over 291 days.

Yikes.

So we need to optimize and optimize we shall. Luckily, there are a few simple performance improvement that can be done to reduce the migration runtime from a year to an overnight success.

Minimize number of connects

Connecting and disconnecting to a database is not free. Many people don’t think about it, but those connections (especially over network) costs resources. It is usually a minimal overhead, but overhead nonetheless. If you connect and disconnect with every query, your overhead will grow in arithmetic progression.

What you want to do, is to maintain the connection for the duration of the script.

// connect to source and target databases
$dbh = new DB(…);
$dbh_trgt = new DB(…);

$max_size = 4200479;

for ($i = 0; $i <= $max_size; $i++) {
  // get and process row
  $sth = $dbh->prepare("select * from source_table where id = ?”);
  $sth->execute($i);

  _process_row($sth->fetch());

  // store row
  $sth_trgt = $dbh_trgt->prepare(insert into target_table ... ”);
  $sth_trgt->execute(...);
}

Condensing 8,400,958 (4,200,479 x 2 databases) connects and disconnects into 2 makes an immediate impact on overall processing time.

Minimize number of queries

A standard solution to n+1 problem is to consolidate all the queries into one (or close to it). Theoretically, that can be done in this migration as well. Instead of selecting individual rows, we can select the whole dataset and then process returned rows in a loop. Unfortunately, returning millions of rows, storing them in memory and processing them at the same time is not good answer. Fortunately, there is a middle ground.

Instead of returning one row at a time (or all the rows) we can return sets of data in a reasonably sized batches.

// connect to source and target databases
$dbh = new DB(…);
$dbh_trgt = new DB(…);

$batch_start = 0;
$batch_end = 100000;
$max_size = 4200479;

while($batch_start <= $max_size) {
  // get rows in batch
  $sth = $dbh->prepare("select * from source_table 
                         where id is between ? and ?”);
  $sth->execute($batch_start, $batch_end);

  // increment batch
  $batch_start = $batch_end+1;
  $batch_end += 100000;

  foreach ($row as $sth->fetchAll()) {
    _process_row($row);

    // store row
    $sth_trgt = $dbh_trgt->prepare(insert into target_table ... ”);
    $sth_trgt->execute(...);
  }
}

The size of batches can vary based on your environment and resource availability. By combining the selected rows into batches of 100,000 we’ve effectively reduced the number of select queries from 4+ million to 43.

Minimize number of transactions

To continue the theme of minimizing the number of round trips to the database, reduce the number of insert transactions sent to the database. Note, transactions not queries. Wrapping multiple queries in a transaction allows to submit multiple (read: thousands) queries in one go. This can be accomplished by either turning AutoCommit flag off and managing batch commits manually, or using BEGIN block to build own batch transaction.

// connect to source and target databases
$dbh = new DB(…);
$dbh_trgt = new DB(…);

$batch_start = 0;
$batch_end = 100000;
$max_size = 4200479;

while($batch_start <= $max_size) {
  // get rows in batch
  $sth = $dbh->prepare("select * from source_table 
                         where id is between ? and ?”);
  $sth->execute($batch_start, $batch_end);

  // increment batch
  $batch_start = $batch_end+1;
  $batch_end += 100000;

  // begin transaction
  $dbh->beginTransaction();

  foreach ($row as $sth->fetchAll()) {
    _process_row($row);

    // store row
    $sth_trgt = $dbh_trgt->prepare(insert into target_table ... ”);
    $sth_trgt->execute(...);
  }

  // commit transaction
  $dbh->commit();
}

Adding all inserts for the batch into a single transaction would send all the queries to the database at the same time to execute and commit. So instead of 100,000 commits going to the database, you only send the transaction with all the insert statements and commit once, also effectively reducing a total number of insert transactions for the migration from 4+ million to 43.

Maximize parallelism

If database is the biggest offender in the 6 second processing time, the techniques above would likely give the script a significant enough boost in performance to complete in a reasonable timeframe. However, if the increase in performance on the database side is not enough or the bottleneck is in the _process_row() function — you can fork your script or run multiple instances of the script in parallel, processing multiple batches at the same time.

If transactions and batching are implemented properly, you can fork multiple processes inserting millions of records at the same time in parallel. This, of course, if limited by CPU (don’t fork bomb the machine!), but even with small number of parallel processes you’re significantly reducing time to completion.

// connect to source and target databases
$dbh = new DB(…);
$dbh_trgt = new DB(…);

$batch_start = 0;
$batch_end = 100000;
$max_size = 4200479;

while($batch_start <= $max_size) {

  //Fork a process
  $pid = pcntl_fork();

  //if a child - process a batch
  if (!$pid) {

    // get rows in batch
    $sth = $dbh->prepare("select * from source_table 
                         where id is between ? and ?”);
    $sth->execute($batch_start, $batch_end);

    // increment batch
    $batch_start = $batch_end+1;
    $batch_end += 100000;

    // begin transaction
    $dbh->beginTransaction();

    foreach ($row as $sth->fetchAll()) {
      _process_row($row);

      // store row
      $sth_trgt = $dbh_trgt->prepare(insert into target_table .. ”);
      $sth_trgt->execute(...);
    }

    // commit transaction
    $dbh->commit();

    //Die to exit a child process
    die();

  }
}

This final version of the migration script will spawn 43 children to run batches of 100,000 records in parallel, finishing in minutes/hours (depending on the processing time and complexity of queries). You can control both the size of batches and the number of parallel processes that best match the environment you’re running it on.

There are always additional performance improvements that could be done, but for most throwaway scripts these techniques should be enough to bring the migration time to a much more reasonable (read: acceptable) level.

Migrations: Getting there faster was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

One of the biggest performance killers happen to be the non-essential actions that are being executed on a critical path of your application. Let’s examine a simple registration example:

router.post('/register', function(req, res){
    var user = new User(req.body);
    user.save( function(err) {
        if (!err) {
            user.savePreferences(req.pref);
            user.sendWelcomeEmail();
            res.render(‘first_time_welcome',
              { title: 'Welcome' + user.name, errors: [err] }
            );
        } 
    });
});

The code looks pretty straight forward. User submits the registration form, the code saves used details and preferences, sends out a welcome email and shows the user a “welcome” page.

Optimizing for user experience — you want to display the confirmation page as soon as possible, so the user can start shopping/posting/doing whatever is most beneficial to your business. As such, you want to make sure that the registration process is fast and painless. So let’s time it and make sure that it is. On average, the code above returned the desired “welcome” page in 2.7 seconds, with occasional spikes taking up to 10 seconds. Not exactly within acceptable performance range, especially since the code is pretty slim, doing two quick database inserts (for user.save and user.savePreferences) and is not doing any heavy logical computation.

Adding timing to individual actions, it appears that the function that takes up the most cycles is user.sendWelcomeEmail.

User.sendWelcomeEmail = function(callback){
  if (!do_not_email(user.email) {
    var email = new Email();
    email.subject = "Welcome " + user.firstname;
    email.cta = getDealoftheDay();
    email.secondary = getOffers(user);
    email.generateBody(user, function() {
      email.send();
    });
  }
};

Bingo! user.sendWelcomeEmail retrieves information from the database, does real-time targeting, and invokes to be sent out immediately. There are likely some optimizations that can be done to the code to speed up the process (with caching, pre-processing and other techniques), but the real question that any developer should be asking themselves is — why are we sending a welcome email during the registration process?

Does it have to be done now?

To answer this question you have to understand what the word “now” means to your business. Should the welcome email be sent to a new user upon registration? Yes. Should the welcome email be sent before user is shown a “welcome” page? Probably not.

Anything not pertinent to the immediate business outcome should not be in the critical path

The immediate goal is to take user into the site . Welcome email, while maybe be just as valuable from the business return perspective, is a secondary goal based on timing. Which means user.sendWelcomeEmail function does not belong in the registration process. And this applies to any other functionality that does not assist in accomplishing the primary goal. Anything not pertinent to the immediate business outcome should not be in the critical path. Which means that all the other processes need to be moved outside of the critical path.

Enter queuing

Queueing is also a good way to queue up a number of actions that can be processed at a pace slower than realtime user experience. The application would add an entry to your queue and external job would pick up those entries in the order of appearance (FIFO).

The are a number of ways (and tools) to implement a queue, but most commonly used ones are MQ-specific tools and databases.

Message Queue Tools

There are tools available to use as a basis for your queue. If you’re reliant on Open Source tools, RabbitMQ is probably the most popular and easy to set up tool. If you’re heavily invested into Amazon ecosystem, SQS may be something to look into.

Database

You can also implement your own queue using any database. Traditionally, you would create a queue table and have a cronjob pull the new data every so often (based on your latency requirements) and process it. Additionally, some of the modern databases implement queueing functionality, allowing for jobs to subscribe to your queue table and wait for new entries, instead of traditional periodic pulls.

The right way

Going back to the registration code, it would require a minor change to the flow.

router.post('/register', function(req, res){
    var user = new User(req.body);
    user.save( function(err) {
        if (!err) {
            user.savePreferences(req.pref);
            sendToQueue(‘welcome email’,user);
            res.render(‘first_time_welcome',
              { title: 'Welcome' + user.name, errors: [err] }
            );
        } 
    });
});

Additionally, your cronjob just need to iterate through the new entries in queue and call the same user.sendWelcomeEmail function that was previously in the critical path of our registration process.

while (var user = readFromQueue('welcome email')) {
   user.sendWelcomeEmail();
}

Now, the welcome emails goes out within 1 to 5 minutes of user registration, which is an acceptable latency, given that the time to “welcome” page has been cut down to 300 milliseconds.

Queueing: Optimizing for critical path was originally published in Web Performance for Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.