Thursday, May 21, 2015

Write hookscripts using Python


hookscript supports Python!

Since we opened for private beta, Python has been our most requested feature.

You can now use Python to run a web app, consume an API, or respond to webhooks. Please share your use cases in the comments below. We love to hear how users utilize hookscript.

Read the Python Documentation for hookscript. Then write your own code.

Want to speed up support for your favorite language? Visit our open source language support page.

Friday, May 1, 2015

Case Study: Mileage Log for Work Using Automatic Webhooks

SUMMARY:
  • Automatically log kilometers driven for work & personal use
  • Script runs only when a trip is finished
  • Respond to webhooks generated by Automatic


BACKGROUND:
Kenny uses his personal car for some work trips. He needs to keep track of work driving vs personal driving in order to claim tax deduction.

PROBLEM:
Manually keeping a log is a pain and he always forgets. Kenny wants to streamline the process and make it more accurate.

ANSWER:
Use Automatic dongle to monitor car trips. Uses hookscript to determine if trip was work/personal and store the data.

STACK:
Code is written in Perl. Car monitoring and webhook generated by Automatic. Data stored with hookscript's built-in persistent state.

Kenny owns an Automatic dongle to monitor his driving. Automatic can generate a webhook (post data to a url of your choosing) anytime a trip is finished. The webhook contains data on start location, end location, and date/time of trip.

The trip JSON data looks like this:
{"trip":
    {"id":"T_79f8283e16a1f288",
        "start_location":{
            "lat":39.100000,
            "lon":-105.100000,
            "accuracy_m":73.45800018310547
        },
        "end_location":{
            "lat":39.000000,
            "lon":-105.000000,
            "accuracy_m":6.0
        },
        "start_time":1429305158409,
        "end_time":1429305470878,
        "start_time_zone":"America/Boise",
        "end_time_zone":"America/Boise",
        "distance_m":2802.3056640625
    }
}
After coding, hookscript provides a url. Any requests to that url run Kenny's code. Kenny uses his script's url as Automatic's webhook destination.
https://www.runhook.com/kenny_unique_script_id_here
The script receives the data via HTTP Post and extracts the important info like start and end location, day of the week, and driving distance.
use Hookscript;
use JSON::XS qw( decode_json );

if ($req->method eq 'POST') {
    # automatic webhook sends POST request to this script's url
    # with JSON data with trip details
    # if script receives a POST request process the trip

    my $json = $req->content->asset->slurp;       # grab JSON
    my $d = decode_json($json);                   # decode it
    my $start = $d->{'trip'}->{'start_location'}; # convenience for repeated usage
    my $end =   $d->{'trip'}->{'end_location'};   # convenience for repeated usage

    # starting latitude and longitude
    # round to 4 decimal places to keep location accurate but allow
    # slight differences depending on where car is parked (garage, street, etc)
    my $start_lat = sprintf('%.4f', $start->{'lat'});
    my $start_lon = sprintf('%.4f', $start->{'lon'});

    # ending latitude and longitude
    my $end_lat = sprintf('%.4f', $end->{'lat'});
    my $end_lon = sprintf('%.4f', $end->{'lon'});

    # meters driven for this trip
    my $meters_driven = $d->{'trip'}->{'distance_m'};

    # day of week trip started.
    my $start_time = $d->{'trip'}->{'start_time'};
    my $weekday = (localtime($start_time))[6]; # 6 is weekday portion of time
For Kenny, all driving on the weekend is for personal reasons. And the IRS doesn't allow deductions for driving to or from work (that's considered commuting instead of work driving).
    # home latitude and longitude rounded for privacy
    my $home_lat = 39.0000;
    my $home_lon = -105.0000;
    my $trip_type;

    # work driving is driving that didn't start or end at home
    my $starts_at_home = $home_lat == $start_lat && $home_lon == $start_lon;
    my $ends_at_home   = $home_lat == $end_lat && $home_lon == $end_lon;

    # and took place during a weekday
    my $is_weekend     = $weekday == 6 || $weekday == 0;

    if ( $starts_at_home || $ends_at_home || $is_weekend ) {
        $trip_type = 'not_for_work';
    } else {
        $trip_type = 'for_work';
    }
hookscript has built-in data storage using persistent state.

After each trip is classified, store the meters driven into 'work' or 'not work' categories.
    # use trip_type as the key for storing meters driven
    # add meters driven to previous total
    $state->{$trip_type} += $meters_driven;
Kenny needs to access the data every year for his taxes. A GET request to the same unique url will show the current data.
} else {
    # use a GET request to show current data

    # display data as HTML
    $res->headers->content_type('text/html');

    # convert meters into kilometers with 2 decimals
    my $km_for_work = sprintf('%.2f', $state->{'for_work'}/1000);
    my $km_not_for_work = sprintf('%.2f', $state->{'not_for_work'}/1000);

    print "<h1>KMs driven for work</h1>\n";
    print "<p>For work: $km_for_work</p>\n";
    print "<p>Not for work: $km_not_for_work</p>\n";
}
Sample of results:


Using hookscript, Kenny was able to:
  • Accurately log distance driven for every trip
  • Forget about recording the log data manually
  • Stop tallying the data for taxes every year


Complete code:
use Hookscript;
use JSON::XS qw( decode_json );

if ($req->method eq 'POST') {
    # automatic webhook sends POST request to this script's url
    # with JSON data with trip details
    # if script receives a POST request process the trip

    my $json = $req->content->asset->slurp;
    my $d = decode_json($json);
    my $start = $d->{'trip'}->{'start_location'};
    my $end =   $d->{'trip'}->{'end_location'};

    # starting latitude and longitude
    # round to 4 decimal places to keep location accurate but allow
    # slight differences depending on where car is parked at home (garage, street, etc)
    my $start_lat = sprintf('%.4f', $start->{'lat'});
    my $start_lon = sprintf('%.4f', $start->{'lon'});

    # ending latitude and longitude
    my $end_lat = sprintf('%.4f', $end->{'lat'});
    my $end_lon = sprintf('%.4f', $end->{'lon'});

    # meters driven for this trip
    my $meters_driven = $d->{'trip'}->{'distance_m'};

    # day of week trip started.
    my $start_time = $d->{'trip'}->{'start_time'};
    my $weekday = (localtime($start_time))[6]; # 6 is weekday portion of time

    # home latitude and longitude rounded for privacy
    my $home_lat = 39.0000;
    my $home_lon = -105.0000;
    my $trip_type;

    # work driving is driving that didn't start or end at home
    my $starts_at_home = $home_lat == $start_lat && $home_lon == $start_lon;
    my $ends_at_home   = $home_lat == $end_lat && $home_lon == $end_lon;

    # and took place during a weekday
    my $is_weekend     = $weekday == 6 || $weekday == 0;

    if ( $starts_at_home || $ends_at_home || $is_weekend ) {
        $trip_type = 'not_for_work';
    } else {
        $trip_type = 'for_work';
    }

    # use trip_type as the key for storing meters driven
    # add meters driven to previous total
    my $prev_meters = $state->{$trip_type};
    $state->{$trip_type} = $prev_meters + $meters_driven;

} else {
    # if GET request show current data

    # display data as HTML
    $res->headers->content_type('text/html');

    # convert meters into kilometers with 2 decimals
    my $km_for_work = sprintf('%.2f', $state->{'for_work'}/1000);
    my $km_not_for_work = sprintf('%.2f', $state->{'not_for_work'}/1000);

    print "<h1>KMs driven for work</h1>\n";
    print "<p>For work: $km_for_work</p>\n";
    print "<p>Not for work: $km_not_for_work</p>\n";
}

Tuesday, April 28, 2015

Documentation is Live with Code Examples


Hookscript's documentation is live!

All the documentation includes code examples using your preferred language.

Use the language drop-down on any page to change your preference.

The documentation covers basics like Hello World. And important hookscript features like handling incoming HTTP requests, using persisent state for data storage, and making outgoing network requests.

There's documentation for debugging using hookscript's full request logs.

Our documentation is open source, so please submit pull requests and create issues if you notice anything we need to fix or would like to see added.

Tuesday, March 31, 2015

Open for Private Beta

Today we launched hookscript for private beta. Right now only people with an invitation code can create accounts and write/deploy code. Everyone else can browse the rest of the site to read more about the service.

hookscript let's you deploy web apps and other scripts with less overhead. Your code can be running in seconds with no need to create an account (except during beta phase), configure a database, or manage servers.

Every user gets 10 minutes of free run time per month (time your cod is executing or compiling).

You can put your name on the wait list for an invitation to the private beta. All users on the wait list will get invitations in the near future but we don't have an exact timeline at this time.

If you have any questions about the site or feedback for us please let us know.

Friday, March 13, 2015

Case Study: Web Form to Store Phone Numbers and Send Texts

SUMMARY:
  • Store a list phone numbers without managing a database.
  • Run web app from anywhere with internet access.
  • Send text messages in bulk with Twilio API.


BACKGROUND:
Kinsey is a member of the PTA (Parent Teacher Association) at her children's school and in charge of reminding everyone about meetings.

PROBLEM:
Sending text messages to all the people on the PTA is a pain. She wanted a better way to send the messages and better way to manage the list.

ANSWER:
Using hookscript's built in data store, Kinsey created a simple html form to add/remove people and send them texts.

STACK:
Written in Perl. Sends text via Twilio. Data storage with hookscript's built in 'State'.

Kinsey wants to use a web form to add people, remove people, and send message. The form is built into the script. GET displays the form. POST changes the data stored in State.

If the HTTP method is "POST" look at the action value for the form and take that action. Either adding or removing data or sending a text message.

# If method is POST do something
if($req->method eq 'POST') {
    given($req->param('action')) {
        when(undef) {
            # die if no action chosen
            die "Need 'action' parameter";
        }
        when('add') {
            # add a new phone number if you choose that action
            push @$state, $req->param('phone'); 
        }
        when('del') {
            # delete the phone number if you choose that action
            my $phone = $req->param('phone');
            $state = [ grep { $_ ne $phone } @$state ];
        }
        when('sms') {
            # send a text message if you choose that action
            my $message = $req->param('message');
            for my $recipient (@$state) {
                send_text_message($recipient, $message);  
            }
        }
    }
}
Display a web form so you can see all phone numbers in the list and take the action you want.
# HTML to display the webform
# web form displays when runhook.com url is visited
$res->headers->content_type('text/html');
print <<'HTML';
<html>
    <head>
        <title>PTA Contact List</title>
    </head>

    <body>
        <h1>PTA Contact List</h1>
        <table>
            <tr>
                <th>Phone number</th>
                <th></th>
            </tr>
HTML

# display a list of all phone numbers.
# clicking delete removes it from list
for my $phone_number (@$state) {
    my $form = qq{
        <form method=POST>
            <input type=hidden name=action value=del />
            <input type=hidden name=phone value="$phone_number" />
            <input type=submit value="Delete" />
        </form>
    };
    # one row for each phone number
    print "<tr><td>$phone_number</td><td>$form</td></tr>";
}

print <<'HTML';
        </table>

        <!-- add a new user to the list -->
        <h2>Add a new number</h2>
        <form method="POST">
            <input type=hidden name=action value=add />
            <input type=tel name=phone autofocus />
            <input type=submit value="Add a phone number" />
        </form>

        <!-- write a text message and send it to everyone on list -->
        <h2>Send a message</h2>
        <form method=POST">
            <input type=hidden name=action value=sms />
            <textarea name=message cols=40 rows=3></textarea>
            <input type=submit value="Send SMS" />
        </form>
    </body>
</html>
HTML
The 'send_text_message' function uses Twilio's REST API to send text messages to everyone on the list.
# the function to send a text message via twilio
# receives one argument: phone number
sub send_text_message {
    my ($to_phone_number, $message) = @_;
    my $from_phone_number = "your_phone_number";
    
    # Stuff for verifying your account with Twilio.
    # Your account info would replace these placeholders
    my $url = "api.twilio.com/2010-04-01/Accounts";
    my $accountsid = "your_account_id_from_twilio";
    my $authtoken = "your_auth_token_from_twilio";
    
    # Build the full API url with account ID & authentication
    $url = "https://" .
           $accountsid .
           ":" .
           $authtoken .
           "\@" .
           $url .
           "/" .
           $accountsid .
           "/Messages";
    
    # Send the POST details to Twilio.  API must include From, To, and Body
    my $res = HTTP::Tiny->new->post_form($url, [
        From => $from_phone_number,
        To => $to_phone_number,
        Body => $message ]);
    
    # If everything went as planned, confirm it
    if ( $res->{status} == 200 || $res->{status} == 201 ) {
        print "SMS sent: $to_phone_number\n";
    }
    # Oops, something went wrong, confirm that too
    else {
        print "FAILED: $to_phone_number => " . $res->{status};
    }
}
Using hookscript, Kinsey was able to:
  • Store a list phone numbers without managing a database.
  • Send text messages in bulk.
  • Send the text messages from anywhere with internet access.
Complete code:
use Hookscript;
use HTTP::Tiny;
use experimental qw( switch );

# initialize state if it's empty
$state = [] if not defined $state;

# If method is POST do something
if($req->method eq 'POST') {
    given($req->param('action')) {
        when(undef) {
            # die if no action chosen
            die "Need 'action' parameter";
        }
        when('add') {
            # add a new phone number if you choose that action
            push @$state, $req->param('phone'); 
        }
        when('del') {
            # delete the phone number if you choose that action
            my $phone = $req->param('phone');
            $state = [ grep { $_ ne $phone } @$state ];
        }
        when('sms') {
            # send a text message if you choose that action
            my $message = $req->param('message');
            for my $recipient (@$state) {
                send_text_message($recipient, $message);  
            }
        }
    }
}

# HTML to display the webform
# web form displays when runhook.com url is visited
$res->headers->content_type('text/html');
print <<'HTML';
<html>
    <head>
        <title>PTA Contact List</title>
    </head>

    <body>
        <h1>PTA Contact List</h1>
        <table>
            <tr>
                <th>Phone number</th>
                <th></th>
            </tr>
HTML

# display a list of all phone numbers.
# clicking delete removes it from list
for my $phone_number (@$state) {
    my $form = qq{
        <form method=POST>
            <input type=hidden name=action value=del />
            <input type=hidden name=phone value="$phone_number" />
            <input type=submit value="Delete" />
        </form>
    };
    # one row for each phone number
    print "<tr><td>$phone_number</td><td>$form</td></tr>";
}

print <<'HTML';
        </table>

        <!-- add a new user to the list -->
        <h2>Add a new number</h2>
        <form method="POST">
            <input type=hidden name=action value=add />
            <input type=tel name=phone autofocus />
            <input type=submit value="Add a phone number" />
        </form>

        <!-- write a text message and send it to everyone on list -->
        <h2>Send a message</h2>
        <form method=POST">
            <input type=hidden name=action value=sms />
            <textarea name=message cols=40 rows=3></textarea>
            <input type=submit value="Send SMS" />
        </form>
    </body>
</html>
HTML

# the function to send a text message via twilio
# receives one argument: phone number
sub send_text_message {
    my ($to_phone_number, $message) = @_;
    my $from_phone_number = "your_phone_number";
    
    # Stuff for verifying your account with Twilio.
    # Your account info would replace these placeholders
    my $url = "api.twilio.com/2010-04-01/Accounts";
    my $accountsid = "your_account_id_from_twilio";
    my $authtoken = "your_auth_token_from_twilio";
    
    # Build the full API url with account ID & authentication
    $url = "https://" .
           $accountsid .
           ":" .
           $authtoken .
           "\@" .
           $url .
           "/" .
           $accountsid .
           "/Messages";
    
    # Send the POST details to Twilio.  API must include From, To, and Body
    my $res = HTTP::Tiny->new->post_form($url, [
        From => $from_phone_number,
        To => $to_phone_number,
        Body => $message ]);
    
    # If everything went as planned, confirm it
    if ( $res->{status} == 200 || $res->{status} == 201 ) {
        print "SMS sent: $to_phone_number\n";
    }
    # Oops, something went wrong, confirm that too
    else {
        print "FAILED: $to_phone_number => " . $res->{status};
    }
}

Friday, March 6, 2015

Case Study: Migrate Web Scraper from dotCloud

SUMMARY:
  • Migrate existing code in less than 2 minutes
  • No need to mess with servers at all
  • Hookscript automatically scales and adds redundancy


BACKGROUND:
PriceCharting.com tracks prices for every video game. Some of this data is scraped from various websites.

PROBLEM:
Some scraping required scaling as product lists grew and debugging as sites changed HTML. dotCloud didn't provide enough logging detail and required choosing the number of servers.

ANSWER:
PriceCharting migrated the code to hookscript for improved logging with no need to manage servers.

STACK:
Written in Prolog. Scrapes HTML using XPath. Outputs data in JSON.

% built in modules
:- use_module(library(hookscript)).
:- use_module(library(debug), [assertion/1]).
:- use_module(library(dcg/basics), [integer//1]).
:- use_module(library(http/json), [json_write_dict/2]).
:- use_module(library(web), []).
:- use_module(library(xpath)).

hook :-
    % fetch the GameStop product page
    req:param(id, Id),
    format(string(Url),"http://www.gamestop.com/-/games/-/~d",[Id]),
    web:get(Url,[status_code(200),html5(Dom)]),
        

Migrating the code took roughly 2 minutes. Cut/paste code and add one line:
:- use_module(library(hookscript)).

Like all HTML scrapers, they can break when underlying HTML changes. Hookscript makes debugging easier because script logs show full HTTP response and incoming HTTP request.

# extract price from html
price(Dom, Condition, Price) :-
    gamestop_condition(GameStopCondition, Condition),
    xpath(Dom, //div(h2/strong(text=RawCondition))/h3(text), RawPriceAtom),
    normalize_space(atom(GameStopCondition), RawCondition),
    normalize_space(codes(PriceCodes), RawPriceAtom),
    phrase(currency(Price),PriceCodes),
    !.
price(_, _, 0).  % a missing price is ok
        

Request Log hookscript request logs

PriceCharting doesn't have to worry about the number of servers, redundancy, or scaling. Hookscript takes care of all that and bills only for runtime consumed.

Case Study: Run Cron Jobs in the Cloud

SUMMARY:
  • Import any external library or module
  • Run scripts on a regular schedule or via HTTP request


BACKGROUND:
JJGames.com ships video games around the world. A script runs on a regular basis to check each order for potential credit card fraud.

PROBLEM:
The person who codes the scripts is different than the server admin. This caused delays in code changes being pushed to production.

ANSWER:
JJGames migrated the code to hookscript where it could be updated remotely.

STACK:
Written in Perl. Uses an API from MaxMind and data stored in an PostgreSQL database.

Perl on hookscript has some common CPAN modules built in, including DBI. Accessing the database was all cut/paste from existing code.

# all perl code needs this on hookscript
use Hookscript;

# CPAN modules pre-installed on hookscript
use DBI;
use DBI qw(:sql_types);

# accessing db is easy
my $data_source = "datasource_info";
my $username    = "username";
my $password    = "password";
my $dbh         = DBI->connect( $data_source, $username, $password);
        

MaxMind offers a module on CPAN to make it easier to access their API.
Using App::fatten, JJGames combined their script code and all its module dependencies into a single file. They uploaded that file to hookscript for execution.

# this is automatically generated with app::fatten
BEGIN {
my %fatpacked;

$fatpacked{"Business::MaxMind::CreditCardFraudDetection.pm"} = ...
  package BusinessMaxMindCreditCardFraudDetection;
...
...

unshift @INC, bless \%fatpacked, $class;
  }
# end of fatpack code

# adding module is standard after fatpack
use Business::MaxMind::CreditCardFraudDetection;

my $ccfs = Business::MaxMind::CreditCardFraudDetection->new(
    isSecure => 1,
    debug    => 0,
    timeout  => 10
);
        

Migrating the code took about 15 minutes.

Now, changes are made on hookscript and instantly pushed live with no need to wait for the server admin.