To explore the tech world, though mostly focusing on iOS, Django, and Drupal development Sometimes it isn’t the perfect that gets in the way of the good, but the revolutionary gets in the way of the evolutionary.

Speeding Up Auto-completes

With the advent of Google search auto-complete, it seems like auto-complete should be everywhere.

Any site that has masses of information that is exposed through various search widgets has considered auto-complete in various stages of product design and development. In light of this, Drupal has allowed for quick auto-completion inclusion for over 6 years. In Drupal 7 it looks a bit like this (search for full guides for complete walkthroughs).

This example is filling in city, state combos in a search field.

/** Implementation of hook_menu **/

function example_menu() {
  
  $items['city_autocomplete'] = array(
    'page callback' => 'autocomplete_city',
    'access arguments' => array('access content'),
  ); 
  return items;
}

function autocomplete_city($string){
  $string = array(':s' => $string . '%');
  $result = db_query('SELECT DISTINCT city, province FROM {location} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    $items[$location->city .', '.$location->province;] = $location->city .', '.$location->province;
  }
  print drupal_json_encode($items);
}

/** Some form alter, or form generation function **/
...

    $form['search_string']['#autocomplete_path'] =  'city_autocomplete';
    $form['search_string']['#attributes']['class'][] = 'form-autocomplete';
...

After this, the field now will issue auto-complete suggestions as the user types.

Not Fast Enough

BUT, our business people were down on the speed that the results came back. They wanted it to go faster. So I decided strip out as much of the Drupal bootstrap as possible, and then bypass completely the index.php loading of the normal Drupal framework.

The goal was to do this while still being able to use the #autocomplete_path form enhancement, no need reinvent everything. To do this, we actually have to create a new base script and point the #autocomplete_path paths to that script. It was easiest to do this by placing the scripts in the same root directory as the regular Drupal index.php script is found.

Let call it city_autocomplete.php. Below is the adjusted module code.

Note that the hook_menu item has the '.php' in it. It still works as that is a valid Drupal menu name.

/** Implementation of hook_menu **/

function example_menu() {
  
  $items['city_autocomplete.php'] = array(
    'page callback' => 'autocomplete_city',
    'access arguments' => array('access content'),
  ); 
  return items;
}

//Technically not needed anymore, but leaving in for posterity.
function autocomplete_city($string){
  $string = array(':s' => $string . '%');
  $result = db_query('SELECT DISTINCT city, province FROM {location} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    $items[$location->city .', '.$location->province;] = $location->city .', '.$location->province;
  }
  print drupal_json_encode($items);
}

/** Some form alter, or form generation function **/
...

    $form['search_string']['#autocomplete_path'] =  'city_autocomplete.php';
    $form['search_string']['#attributes']['class'][] = 'form-autocomplete'; 
...

Now, inside the new city_autocomplete.php file. Much of this is taken directly from the index.php file, but we restrict the drupal bootstrap to only the database, and we include one extra file, common.inc which has the drupal_json_encode() function.

<?php

  define('DRUPAL_ROOT', getcwd());
  
  require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
  require_once DRUPAL_ROOT . '/includes/common.inc';
  drupal_bootstrap(DRUPAL_BOOTSTRAP_DATABASE);
  
  // arg(1) is used to gather the data sent to the auto complete script
  $string = array(':s' => arg(1) . '%');
  
  // The rest is just the body of the autocomplete_city($string) function
  $result = db_query('SELECT DISTINCT city, state FROM {zipcodes} WHERE city LIKE :s order by city limit 20', $string);

 
  $items = array();
  foreach ($result as $location) {
    //$items[$location->city] = $location->city .', '.$location->province;
    $items[$location->city .', '.$location->state] = $location->city .', '.$location->state;
  }
  print drupal_json_encode($items);

Clear your cache to reload the menu items, and revisit whatever page had your auto-complete attached to it. That is all.

Benchmarking (all times from Firebug Console)

I wouldn't call this a strenuous set of benchmarking, but does give the ballpark opportunities for speed improvements.

Local system (vanilla MAMP setup, no code caching):

Regular Drupal auto complete: 250-350ms
Limited Bootstrap auto complete: 35-55ms

Dramatic improvement on a vanilla Apache setup, no code caching.

Remote Pantheon System (Pantheon DROPS, http://helpdesk.getpantheon.com/customer/portal/articles/361254):

Regular Drupal auto complete: 150-250ms
Limited Bootstrap auto complete: 80-150ms

First take-away, the regular auto-complete was faster under a remote Pantheon server, than my vanilla localhost. Obviously using a code cache is a great performance booster for the regular code execution.

Second take-away, this method does improve performance speeds by around 20-40%, even under great code conditions.

Bonus tip

You can decrease the delay time between the last key stroke and when the search starts. It defaults to 300ms (essentially doubling the times to display results). It is found in the /misc/autocomplete.js file

...

Drupal.ACDB = function (uri) {
  this.uri = uri;
  this.delay = 300;
  this.cache = {};
};
...

Change that delay to something less to make all auto-completes start faster. If you don't want to do a core hack, then you need to remake all the Drupal.ACDB JS found in the /misc/autocomplete.js in your own theme or module JS, and make the adjustment there.

Quick Tips for Django and Heroku, Static Files and Multiple Environments

For a quick publicly facing Django prototype I decided to use Heroku.com. For low use, it is free, and they have plenty of good docs to get you started, and masses of add-ons, many of which have free tiers as well.

https://devcenter.heroku.com/articles/django is really all that is needed to get you out of the gate and have something running. This guide walks you through setting up a local environment, using 'pip install' to add a few dependencies needed for Heroku, committing the code to git, and then pushing the code to the Heroku remote, which triggers installing and launching the app.

Instead of reinventing the wheel, I will only highlight some issues I had with that initial setup and how I got around them.

Static File Serving

The Django manage.py runserver command explicitly states that it shouldn't be used in a production environment, and that it should be used to serve static files. Also, Heroku doesn't not allow for static file usage, instead pointing you to S3 or other CDN type services for static files. But for a prototype, I thought that was overkill, so I checked around StackOverflow and other sites for the way to get Heroku to serve Django files for the Heroku filesystem.

To get this to work takes some adjustments in three different places, the settings.py file, the main urls.py file and your Procfile that launches the app in Heroku.

First, you need to setup your Static Files location in a way that works for both your local environment and on Heroku (trying to avoid branching configs per environment).

Generally, there is a "#STATIC_ROOT = ''" type entry in the settings.py file, that more or less says collect all the static files and place them 'somewhere'. Well, in Heroku, this isn't good, as that somewhere can't be absolutely defined, and it would break your local development. Below is the first step, which uses asks the os where to place things relative to the settings.py file.

import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
STATIC_ROOT = os.path.join(PROJECT_ROOT,'staticfiles/')
# URL prefix for static files.
# Example: "http://example.com/static/", "http://static.example.com/"
STATIC_URL = '/static/'

Then in the urls.py file add this entry:

urlpatterns += patterns('',
    (r'^static/(?P.*)$', 'django.views.static.serve', {'document_root': settings.STATIC_ROOT}),
)

Lastly, you need to tell Heroku to actually collect all the static files. The regular 'manage.py runserver' command does this for you in development, but if you followed the above guide you setup a gunicorn process to handle your web requests, and it knows nothing about collecting static files for Django.

In your procfile, replace the entry you have which probably looks like this:

web: gunicorn hellodjango.wsgi

With this:

web: python manage.py collectstatic --noinput; gunicorn hellodjango.wsgi

This runs the collectstic command from the manage.py script, and then launches the gunicorn process.

Now you have a simple static file server bases inside Heroku. It really isn't a good setup for a scalable production environment, but for a quick prototype, it is nice not to have to hook into, and maybe pay for, a CDN or external file system.

One great tip I liked from the Two Scoops of Django book is that you should try to keep your settings.py as environment agnostic as possible, and that means setting and using environmental variables liberally. There are many help pages for settings such variables in Windows or on a Mac, so I don't want to cover that here, but I will touch on how to use them in your Django settings.py file and how to set them in Heroku.

Borrowing from Two Scoops of Django, place this at the top of your settings.py file.

import os
import dj_database_url

from django.core.exceptions import ImproperlyConfigured

def get_env_variable(var_name):
    """ Get the environment variable or return exception """
    try:
        return os.environ[var_name]
    except KeyError:
        error_msg = "Set the %s environment variable" % var_name
        raise ImproperlyConfigured(error_msg)

It imports a few needed libraries, as well as adds a helper function we will use in other settings.

For most developers, there is a distinct set of settings that are specific for local development, so I decided to add a LOCAL_DEV variable, that lives along side of the DEBUG one that is used by Django.

DEBUG = get_env_variable("DJANGO_DEBUG")
if DEBUG == 1 or DEBUG == '1':
    DEBUG = True
else:
    DEBUG = False
    
LOCAL_DEV = get_env_variable("DJANGO_LOCAL_DEV")
if LOCAL_DEV == 1 or LOCAL_DEV == '1':
    LOCAL_DEV = True
else:
    LOCAL_DEV = False

if DEBUG:
    DEBUG_TOOLBAR_PANELS = ( ...settings...   )
    #...other DEBUG settings ....

DATABASES = {}

if LOCAL_DEV:
    ALLOWED_HOSTS = ['*'] #useful when testing with DEBUG = FALSE
    INTERNAL_IPS = ('127.0.0.1',) #sets local IPS needed for DEBUG_TOOLBAR and other items.

    DATABASES = {
        'default': {
            
            'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
            #'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
            'NAME': 'hellodjango',                      # Or path to database file if using sqlite3.
            #'NAME': '/Users/SomeUser/Documents/Django Projects/hellodjango/hellodjango.sql',                      # Or path to database file if using sqlite3.
            # The following settings are not used with sqlite3:
            'USER': '',
            'PASSWORD': '',
            'HOST': 'localhost',                      # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
            'PORT': '',                      # Set to empty string for default.
        }
    }
else:
    # Parse database configuration from $DATABASE_URL
    DATABASES['default'] =  dj_database_url.config()
    
    # Honor the 'X-Forwarded-Proto' header for request.is_secure()
    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

First you might ask, why are those two settings coming in as 0 and 1, instead of True and False. Answer, in Heroku, all the settings are saved as strings, and the string 'False' will return as True, which caused me an hour or two of pain. So I check for the 1 and 0 and change to True and False.

This setup allows for the local development to use a local database, while Heroku uses the DB url through the 'dj_database_url.config()' call to find its database somewhere in the clouds.

Lastly, the writers suggest that the SECRET_KEY be stored as a variable, that way it is never in your source code. I also updated the DEFAULT_FROM_EMAIL, that way I could adjust it without needed to relaunch the Heroku app.

DEFAULT_FROM_EMAIL = get_env_variable("DJANGO_EMAIL_DEFAULT_USER")

SECRET_KEY = get_env_variable("DJANGO_SECRET_KEY")

But we still haven't actually set these variables in Heroku. From the root of your project on your system, you can use the 'heroku config' command to handle it.

$ heroku config
DATABASE_URL:     postgres://....amazonaws.com:....
HEROKU_POSTGRESQL_CYAN_URL:     postgres://....amazonaws.com:....

Since you haven't set any yet, the created DB variables are all that is present. To add a variable use the 'heroku config:add VARNAME=stuff'

$ heroku config:add DJANGO_EMAIL_DEFAULT_USER=from@sample.com
heroku config:add DJANGO_SECRET_KEY=something_secret
heroku config:add DJANGO_DEBUG=1
heroku config:add DJANGO_LOCAL_DEV=0

Now, all the settings are loaded, and you can check again:

$ heroku config
DATABASE_URL:     postgres://....amazonaws.com:....
DJANGO_EMAIL_DEFAULT_USER:  from@sample.com
DJANGO_LOCAL_DEV:           0
DJANGO_DEBUG:               1
DJANGO_SECRET_KEY:          something_secret
HEROKU_POSTGRESQL_CYAN_URL:     postgres://....amazonaws.com:....

Multiple Environments for Apps

Lastly, a couple quick tips regarding having a staging and production environment on Heroku for your app. Since mostly likely everything you are doing thus far is free, double free is still free, so why not spin up a staging server, to both test new settings and make sure you don't break anything just as a client is playing with your prototype.

https://devcenter.heroku.com/articles/multiple-environments has most of the tips you need. Once getting through the article you can now issue heroku commands and append the '--remote staging' to target the staging server, instead of the main production one.

I never liked the 'git push heroku master' command, as it just didn't feel informative enough. So let's first rename that to 'production':

git remote rename heroku production

Now you would run 'git push production master' and 'git push staging master' to push code to the Heroku apps.

Lastly, make sure you run this command, so that any 'heroku' commands that you run without a --remote flag will go to the staging server, instead of production (making it a bit more accident proof).

$ git config heroku.remote staging

Well, this has become a lengthy article, and there are still a few other items I might mention, but I will hold them off to another article. Mostly they had to do with deploying Django generically, not specifically tied to Heroku.

Python and Django

'Sure, why not', I said to myself. To keep extending my skill set in the tech industry is a must, so I decided that as I transition back to webapps from iOS apps, I would learn something new as I created my new project (more on that later).

Instead of jumping back to the familiar PHP/Drupal combination that has served me well for DogGoes.com, RVParking.com and other such properties, I decided to take Python on finally, and one of its web frameworks, Django.

I will write a bit more Django specific posts in the future, but just wanted a nice starting place. This is not going to be a helper post for either Python or Django, just a quick list of some assets I used in the first few weeks as I was getting spun up, and shot in the right direction.

First, the main docs for Django are at https://docs.djangoproject.com/en/1.5/. But as a starter, I worked through all six of the tutorials they have listed there. A good first lap around the track to learn the basics of Django.

Next, I read Two Scoops of Django. A quick primer of tips and best practices, a good read at a great price.

Obviously, I needed some good Python resources as well.
I worked through Python The Hard Way.
OverAPI.com has a Python listing, as well as a ton of other great resources.

Lastly, some quick reference sheets to live on the desk were needed.

Python Cheatsheet by Dave Child
Django Cheat Sheet by RevSys.

Changes Ahead

Things have gone a bit silent here lately. The main reason is that I have been doing an evaluation of what I have been working on and where I wanted to go with my skill set.

While making iOS games for over a year, releasing three different games to varying degrees of success, my heart wasn't in it to continue along that path.

Having now been out of the startup realm of businesses for 18 months, I am starting to feel like I want to get back in that game. I wasn't burned out after T2Media sold AboutAirportParking.com (which had a good financial upside for me), nor during the next year where we worked to replicate the prior success, but I felt I needed a change which is why Skejo Studios was started.

I could work on the projects I wanted, on the timelines I crafted. Of course I had to finance the work that I couldn't do, which brought in a whole new set of skills and management that I yearned to grow in. Which I know I have.

But now, as 2013 is half way through, I am again focusing my development skills toward web applications and away from iOS games. Another time I will write about exactly what I have been doing that last month.

Deadlines, and Indie Experience

Block you menu screenDeadlines. Love them or Hate them. Evil things that demand overtime from underpaid artists and engineers, or the only thing producers can use to nail down those creative types to actually deliver assets.

I don't want to get into the pros or cons of deadlines, I just want to share my perspective as an indie, and how some deadlines helped us ship our latest free to play, iOS title, Block You.

As an indie developer, the 'It ships when it is ready' mentality is very prevalent. Not constrained by typical oversight, with costs less than visible than normal (didn't get paid this week, won't get paid next week, so who cares if it slips a week), it is easy for an indie to keep plugging away at code and art assets, watching the days and week fly by.

That is exactly what happened to me on my first two games. Both missed their original timelines for over 50%. Sure chalk it up to learning curves and features creeps, but really, how can I plan a 6 or 12 months of releases and projects, if what I call a 3 month project really takes me 5 month to do. So here is my take at trying to set some deadlines, add some time pressure into my development cycles and see how I do.

Phases of Level Polish on Lab Bunnies

Having not taken a regular course of level design outside those offered by the experiences of life and trial and effort, I now endeavor to document the phases of level design and polishing that went into Lab Bunnies, now available in the AppStore.

Steps in Brief

  • Hand Drawn Designs
  • Constructed in Level Editor (at times by another person)
  • Game Tested and Tweaked for Playability by original designer (in case the original design was untenable or not fun)
  • Level replayed many times to gauge correct time and movement bonuses
  • Decorations were added (by original designer or another)
  • After all this, a series play testing revealed a some levels that needed further improvement or updates.

You Won't be a Good Manager if Can't Let Someone Go

You are FiredPaging Donald Trump!!! Do indie devs and indie studios stay mediocre or bad because they won't part ways with someone not pulling their weight?

"You Won't be a Good Manager if Can't Let Someone Go" were the words a close friend utter to me in the midst of my latest project. But is it true?

Must you go through the pain of letting someone go before you can really be good at managing a project/studio?

Let's discuss in the comments, because I haven't decided yet, but I will give one main thought on why it happened for me. NOTE: Is related to our upcoming game, Lab Bunnies.

The App Store Playbook - My Interview

Earlier this year I was glad to offer an interview with Shane Lee at Beginning iOS Dev while I making Lab Bunnies. He has packaged my interview, and 8 other indie developers including @DeneCarter Creator of Incoboto, @ADAMATOMIC Creator of Canabalt, and @davekalina Creator of Spider: The Secret of Bryce Manor.

Here is an excerpt of the interview that I participated in:

Lab Bunnies Coming Oct 25th

We at Skejo Studios are glad to announce our debut title Lab Bunnies, which weaves the story of the first, super smart, cloned sheep, crafting their own creations, cloned bunnies. But as they experiment on their own clones, someone is still experimenting on them.

Lab Bunnies is an iPhone/iPod touch top scrolling puzzle platformer. Created for casual players and serious gamers alike, but either way, we are crafting a delightful game.

 

Lab Bunnies, the Facts

  • iPhone 5 ready, along with iPhone 4(S)/iPod Retina and iPhone 3GS resolutions
  • Over 200 sections to complete, presented in 72 levels and 3 different worlds
  • Casual game play, with challenges for the die hard gamers
  • Intuitive one touch controls, with alternative Left/Right/Jump buttons
  • Natural progressions for casual players
  • iCloud progress syncing (get a new iOS device and your progress goes with you

Links:
Teaser Trailer
Screen Shots
Game Link
Purchase (works only on 10/25/2012 and after)

Device Requirements:

  • iPhone 5 ready
  • iPhone 4(S) & 4th Gen iPod Retinal-graphics * Playable on non-retinal devices
  • Requires iOS 4.2 or later
  • 21.0 MB

Operation: Eradicate Financials Three Months In

Operation: Eradicate, has been in the AppStore a bit over three months now, and I wanted to give a report of the ups and downs regarding the financial outlook of the game. Eradicate was a great game for us to create, laying a good foundation for our studio both from production & technical standpoints for which we can launch our newly announced game: Sheepo (A Kickstarter just started to help our artists).

For a bit of background please read about Post Launch Marketing Analysis and our game dev Post Mortem.

Quick Take: Not a Flop, Not a Success

According to a survey last year of indie developers, 50% of first time games don't clear $500, and 75% of indie developers don't make $10,000 off a game in a year.

As of this writing we are a couple days away from $10,000 in NET Revenue (after Apple's cut), which only took three month in the AppStore to hit. Thus regardless of my own personal expectations, we shouldn't call Operation: Eradicate a flop. But until we cover about $20K in net revenue, I can't really call it a break-even game either, as far as time and money spent.

This graph is the fast way to represent where we have been since launch. As can be seen, being listed as a Staff Favorite for iPad games isn't a game changer. It moves the needle a bit while on the list, but doesn't fundamentally change sales like a New & Noteworthy featuring, or a What's Hot notice (two years ago I had a free app featured, 15x downloads for that week).

But we do feel that our quick update to include Retina Display ready images from the new iPad is what placed us on the Staff pick list. Thus continual maintenance of a game, may get rewarded by Apple in the long term. But, other's experience may vary.

Also, we have been playing with the price every couple weeks, moving around from $3.99 to $1.99. The further we get from the initial launch the harder it is to retain revenue at the higher price. We feel now that for our niche Strategy game, the $3.99 was briefly sustainable, but now $1.99 is the highest we can go, and will in the future do periodic 1/2 off sales to $0.99.

Where we are headed now

Besides working on our next game, we will continue to support Operation: Eradicate. We are currently translating it to German, and will do a few other translations next month.

We are weighing the cost of adding in new game modes and possibly charging for them. We are also toying with the idea of setting it free for a few days (not using any paid service, just free on our own) and see how that affects sales. Partly we will do that to establish a baseline if we later try a free app promotion type thing.

I hope some of these facts and figures encourage other indies, or at least show a transparent light on what happens around small iOS studios. Skejo Studios is committed to the long haul. We have much personal and studio financial reserves to keep us going for a while, and a great budding team to keep pressing our game designs as we strive for fun and successful games.

Syndicate content