drupal

Speeding Up Auto-completes

With the advent of Google search auto-complete, it seems like auto-complete should be everywhere.

Any site that has masses of information that is exposed through various search widgets has considered auto-complete in various stages of product design and development. In light of this, Drupal has allowed for quick auto-completion inclusion for over 6 years. In Drupal 7 it looks a bit like this (search for full guides for complete walkthroughs).

This example is filling in city, state combos in a search field.

/** Implementation of hook_menu **/

function example_menu() {
  
  $items['city_autocomplete'] = array(
    'page callback' => 'autocomplete_city',
    'access arguments' => array('access content'),
  ); 
  return items;
}

function autocomplete_city($string){
  $string = array(':s' => $string . '%');
  $result = db_query('SELECT DISTINCT city, province FROM {location} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    $items[$location->city .', '.$location->province;] = $location->city .', '.$location->province;
  }
  print drupal_json_encode($items);
}

/** Some form alter, or form generation function **/
...

    $form['search_string']['#autocomplete_path'] =  'city_autocomplete';
    $form['search_string']['#attributes']['class'][] = 'form-autocomplete';
...

After this, the field now will issue auto-complete suggestions as the user types.

Not Fast Enough

BUT, our business people were down on the speed that the results came back. They wanted it to go faster. So I decided strip out as much of the Drupal bootstrap as possible, and then bypass completely the index.php loading of the normal Drupal framework.

The goal was to do this while still being able to use the #autocomplete_path form enhancement, no need reinvent everything. To do this, we actually have to create a new base script and point the #autocomplete_path paths to that script. It was easiest to do this by placing the scripts in the same root directory as the regular Drupal index.php script is found.

Let call it city_autocomplete.php. Below is the adjusted module code.

Note that the hook_menu item has the '.php' in it. It still works as that is a valid Drupal menu name.

/** Implementation of hook_menu **/

function example_menu() {
  
  $items['city_autocomplete.php'] = array(
    'page callback' => 'autocomplete_city',
    'access arguments' => array('access content'),
  ); 
  return items;
}

//Technically not needed anymore, but leaving in for posterity.
function autocomplete_city($string){
  $string = array(':s' => $string . '%');
  $result = db_query('SELECT DISTINCT city, province FROM {location} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    $items[$location->city .', '.$location->province;] = $location->city .', '.$location->province;
  }
  print drupal_json_encode($items);
}

/** Some form alter, or form generation function **/
...

    $form['search_string']['#autocomplete_path'] =  'city_autocomplete.php';
    $form['search_string']['#attributes']['class'][] = 'form-autocomplete'; 
...

Now, inside the new city_autocomplete.php file. Much of this is taken directly from the index.php file, but we restrict the drupal bootstrap to only the database, and we include one extra file, common.inc which has the drupal_json_encode() function.

<?php

  define('DRUPAL_ROOT', getcwd());
  
  require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
  require_once DRUPAL_ROOT . '/includes/common.inc';
  drupal_bootstrap(DRUPAL_BOOTSTRAP_DATABASE);
  
  // arg(1) is used to gather the data sent to the auto complete script
  $string = array(':s' => arg(1) . '%');
  
  // The rest is just the body of the autocomplete_city($string) function
  $result = db_query('SELECT DISTINCT city, state FROM {zipcodes} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    //$items[$location->city] = $location->city .', '.$location->province;
    $items[$location->city .', '.$location->state] = $location->city .', '.$location->state;
  }
  print drupal_json_encode($items);

Clear your cache to reload the menu items, and revisit whatever page had your auto-complete attached to it. That is all.

Benchmarking (all times from Firebug Console)

I wouldn't call this a strenuous set of benchmarking, but does give the ballpark opportunities for speed improvements.

Local system (vanilla MAMP setup, no code caching):

Regular Drupal auto complete: 250-350ms
Limited Bootstrap auto complete: 35-55ms

Dramatic improvement on a vanilla Apache setup, no code caching.

Remote Pantheon System (Pantheon DROPS, http://helpdesk.getpantheon.com/customer/portal/articles/361254):

Regular Drupal auto complete: 150-250ms
Limited Bootstrap auto complete: 80-150ms

First take-away, the regular auto-complete was faster under a remote Pantheon server, than my vanilla localhost. Obviously using a code cache is a great performance booster for the regular code execution.

Second take-away, this method does improve performance speeds by around 20-40%, even under great code conditions.

Bonus tip

You can decrease the delay time between the last key stroke and when the search starts. It defaults to 300ms (essentially doubling the times to display results). It is found in the /misc/autocomplete.js file

...

Drupal.ACDB = function (uri) {
  this.uri = uri;
  this.delay = 300;
  this.cache = {};
};
...

Change that delay to something less to make all auto-completes start faster. If you don't want to do a core hack, then you need to remake all the Drupal.ACDB JS found in the /misc/autocomplete.js in your own theme or module JS, and make the adjustment there.

Flying Solo Again

It has been a bit since a blog post has rolled out, but I have had my head down finishing some fun Drupal projects. But I intend that to change for a couple reasons.

Book Review: Drupal 7 First Look

Last month I received a ditigal copy of Drupal 7 First Look, by Packt Publishing. I have been waiting for the official release of Drupal 7 to come out before posting the my review and now is the time. The subtitle is Learn the new features of Drupal 7, how they work and how they will impact you, so I will be reviewing the book in that light. Tackling a new version of Drupal in 256 pages is a tall order, so let's see how it all unfolds. This book isn't exactly for Drupal newcomersas as some prior Drupal knowledge is assumed, but if you are completely new you can get by nicely.

Briefly, the author, Mark Noble, owner and operator of Turning Leaf Tech, has been working with Drupal for a couple years. He is also the author of Drupal 6 Site Builder Solutions, and runs http://drupalbyexample.com.

The book broken into seven chapters, progressing from what is new in Drupal, installation/upgrade, initial site building and administration. The last three chapters dive more deeply into the APIs for theme developers, the database layer and for developers.

Chapter 1 goes through a decent list of changes from Drupal 6 to Drupal 7. If you aren't familiar with Drupal this will help you get up to speed about where Drupal came form and where it is going. Chapter 2 hits on the basics of installing Drupal 7 or upgrading from Drupal 6. Many screen shots are include to speed you through the process. If you are familiar the install profile ideas, Mark places some code to show how you can create and use an new install profile. Handy for anyone install numerous sites that need a generic starting place.

Things really pick up in Chapters 3 and 4 as Mark expounds how to create content and new content types. WIth CCK now Fields in core, the Field API is introduces along with images, taxonomies and commenting. CHapter 4 focuses on non-content administration of users management, sites settings, module selection, search, reports and other items.

From Chapter 5 onwards the book takes a more technical bent as more code examples and API code definitions appear. CH5 focuses on new and change theming items including template files, new JavaScript abilities, CSS and Theme API changes.

Chapter 6 hits on the major database update the Drupal 7 introduces. The code samples are great, detailing how to call and step through the results of a database query.

Chapter 7 ends a bit abruptly, as does the whole book after covering the many change and new API hooks available for developers to use. Considering this isn't a cookbook or dive deeply into development I have to say that if you need a crash course is what Drupal 7 can do out of the box and a quick primer on how to drive a Drupal 7 site, this book does fine.

Drupal 7 First might be compared to Foundation Drupal 7 by Apress, though I have read the Apress book yet. If you need more core API and module development insight, try grabbing Drupal 7 Module Development by Packt or Pro Drupal 7 Development by Apress.

A Drupaler's Command Central

It is the new year and I decided I needed to clean off my desk, and since it looked so clean, I snapped a quick picture. I currently am working from home as the company I work for has gone virtual, which is why I have this setup in the first place.


(click for full image)

Items:
*Lenovo T61 ThinkPad 15.4" 1680x1050 screen (company provided w/ docking station)
*24" Dell Monitor 1920x1200 screen (company provided)
*Apple Macbook 14" 1280x800\
*Microsoft Ergonomic Keyboard
*Logitech Thumbscroll mouse
*TrendNet KVM (allows the 24" monitor, keyboard and mouse to be shared by the Thinkpad and MacBook)
*iPad (hard to see, used for Drupal powered iOS app testing, other stuff)
*iPhone 4 (taking the picture)
*iPhone 3 (not pictured, keeping for backward compatibility of iOS apps)
*Brother Printer/Fax/Scanner w/ Drum ink (b/w only but uses less ink long term)
*Whiteboard and PostIts used to keep different projects and priorities in front of me.

The laptop screen is where I keep my mail program and other readable items that I might need as I code.

The main 24" monitor is where the main coding and testing happens.

Hidden behind my Komodo IDE window, is a FF Browser with Drupal Planet. There are a ton of Drupal 7 released announcements right now, in case you missed that.

Other items of interest:
*I am a hockey fan in San Jose, CA, thus the Sharks poster.
*The nearly ever present coffee mug is actually missing.
*Hidden under the laptop stand are snacks and TicTacs.
*Super cheap speakers don't give any bass.

Drupal.org builds a new home

If you haven't seen it yet (ie you are reading Drupal Planet through a feed reader), then head over to Drupal.org check out the new home for the Drupal project.

I am really digging the new personal dashboard that pull together projects, issues, commits, comments and groups all in one place.

Great work to the Design implementers (also a way to check out the new groups.drupal.org look. Much applause to all.

Drupal Userpoints and the big Boys

In the last two weeks I have come across three different venture backed groups that are trying to help sites 'Gamify' their member interactions.

Drupal 7 Beta, kicking tires time

I am willing to admit that I am not cutting edge guy, and neither is the company I work for. I keep up to date on new modules and processes, but rarely try out alpha labeled modules.

Drupal 6 Ultimate Community Site Guide Review

Recently I received a review copy of Drupal 6 Ultimate Community Site Guide (2nd edition) and I jotted down some notes as I went through the book. I am currently constructing a more community focus site, and I was hopeful to learn a few pointers along the way. There both ebook and paperbook versions of the book.

My general impressions is that the book tries to tackle some important site building tasks using Views, Panels, CCK, and other important contributed modules, while also focusing on the using standard core modules (and more that will be in Drupal 7 core) which is important for future proofing the site. The author, Dorien does a good job to use tackle many modules that fill out a community site, but of course you can't tackle them all in 150 pages (no User Badges, cursory mention of blogs, and Organic Groups is not fleshed out much).

Keep reading for a chapter by chapter breakdown of the book.

Drupal for Facebook, Quick posting to FB Streams

In a recent project update to our RV Parks site we wanted to add some Facebook integration points. Add a few 'Like' buttons for Facebook on pages as well as the ability to login using Facebook connect. Numerous modules can do this for Drupal, but we moved forward with Drupa for Facebook (demo site at http://www.drupalforfacebook.org) based on some other features I want to talk about here.

Drupal for Facebook also has Facebook Stream integration in a submodule, which we wanted to use to post reviews written by our users to their FB walls, or streams as they are now called. The Streams module is more of an API that you need to implement in a custom module, but a fine example is given as a recipe in the Drupal for Facebook documentation at http://drupal.org/node/685320. A great example that worked for us with a bit of our own tweaks.

But, what we really want to do is post directly to FB without the extra popup confirmation that is used the example. We wanted to speed up the workflow. To do this we need get an 'Extended Permission' from Facebook for each user. The Extended Permission contrib module that is packaged with Drupal for Facebook exposes these permission, but only on the user's profile page, which wasn't ideal for our use case.

So we need to obtain the extended permission skip the confirmation, and then we need to update the code to bypass the confirmation screen if we do have the permission. To do this we have to update the form_alter hook in the example to ask for the permission, and then update the nodeapi call to skip the confirmation if the permission is given.

Here is the updated form_alter code from the example:

function dff_custom_form_alter(&$form, $form_state, $form_id) {

   if (isset($GLOBALS['_fb']) && fb_facebook_user()) {

   $fb = $GLOBALS['_fb'];
   $fb_app = $GLOBALS['_fb_app'];

   if ($form['#id'] == 'node-form') {
     // Add checkbox to control feed publish.
     $form['dff_custom']['stream_publish'] = array(
       '#type' => 'checkbox',
       '#title' => 'Share on Facebook',
       '#default_value' => TRUE,
     );
   }
   else if ($form['form_id']['#value'] == 'comment_form') {
     // Add checkbox to control feed publish.
     $form['dff_custom']['stream_publish'] = array(
       '#type' => 'checkbox',
       '#title' => 'Share on Facebook',
       '#default_value' => TRUE,
     );
   }

   try {
       $has_perm = fb_call_method($fb, 'users.hasAppPermission', array(
       'ext_perm' => 'publish_stream',
         'uid' => $fbu,
       ));
     } catch (Exception $e) {
     fb_log_exception($e, t('Call to users.hasAppPermission(%key) failed.', array('%key' => $key)));
   }

     //If the permission has not be granted yet, add a link to a new window that asks for the permission.
     if (!$has_perm) {
     $form['dff_custom']['stream_publish']['#description'] =
       l('Auto share all reviews (skip confirmation popup)',
'http://www.facebook.com/authorize.php',
       array('query' => 'api_key='.$fb_app->apikey.'&v=1.0&ext_perm=publish_stream&next=http://www.rvparking.com/user/'.$user->uid.'&next_cancel=http://www.rvparking.com/user/'.$user->uid, 'html' => TRUE, 'attributes' => array('target' => '_blank'))
       );
     }

   }

}

Specifically the try/catch code (couched from the Extended Permissions module) is checking to see if the 'publish_stream' extended permission has already been granted by this user. If not, then a link is added to the description of the 'Share on Facebook' form element.

Next we update the hook_nodeapi function (similar edits needed for comments, not shown here).

/**
* Implementation of hook_nodeapi().
*
* Publish to facebook Walls when users submit nodes.
*/
function dff_custom_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
   if ($op == 'insert' || $op == 'update') {
     if (isset($node->stream_publish) && $node->stream_publish) {
       //dpm($node, "dff_custom_nodeapi, publishing to stream");
       // http://wiki.developers.facebook.com/index.php/Attachment_(Streams)
       $attachment = array(
         'name' => $node->title,
         'href' => url('node/' . $node->nid, array('absolute' => TRUE)),
         'description' => $node->teaser,
       );

       $user_message = t('Check out my latest post on !site...',
       array('!site' => variable_get('site_name', t('my Drupal for Facebook powered site'))));
       $actions = array();
       $actions[] = array('text' => t('Read More'),
         'href' => url('node/'.$node->nid, array('absolute' => TRUE)),
       );

       $fb = $GLOBALS['_fb'];

       $fb_session = $fb->getSession();
       $fb->api_client->session_key = $fb_session['session_key'];
       $fb->api_client->expires = 0;

       try {
         $success_data = fb_call_method($fb, 'stream.publish', array(
           'session_key' => $fb_session['session_key'],
           'message' => $user_message,
           'attachment' => $attachment,
           'action_links' => json_encode($actions),
         ));
       } catch (Exception $e) {
         drupal_set_message("NOTTTT!!! Added on FB Wall: Showing Dialog.");
         fb_stream_publish_dialog(array('user_message' => $user_message,
           'attachment' => $attachment,
           'action_links' => $actions,
           ));
       }
     }
   }

}

Now we use the try/catch method to post the update to Facebook directly. The fb_call_method function is provided by Drupal for Facebook. If the user has granted the 'publish_stream' permission, then the post will succeed, if not then we use the normal fb_stream_publish_dialog method which will create a JS popup box after the form sumbits that will ask the user if they want post to Facebook.

Now we have much more streamlined posting to Facebook workflow. If no extended permission are granted, it behaved just like the example with a popup confirmation. But now in the form we ask for the extended permission, and if granted we speed up the posting to Facebook. Good for all.

Advomatic plus Mercury is Fast

Coming out of the SF DrupalCon Cloud Hosting session, I had been looking for a hosted Mercury solution. I was waiting for Chapter Three's Panteon Mercury solution, but it has been in private beta for months.

Then I saw that Advomatic has launched a hosted Mercury solution, so I checked it out.

I will let the data speak for itself concerning the speed improvement's seen by my private Dog parks website. Here is the graph and it is easy to see when I move the hosting over.

Dog Park USA speed graph

And beyond that, it was very simple to move over the site. Just merged my Drupal 6 install into a Drupal Pressflow install, added and installed the memcache and varnish modules, and watched the load times drop.

Even if your site only has thousands of visitors a day, give them a faster site.

Syndicate content