Tag Archives: javascript

How IFTTT, Mandrill, and Twilio is helping me find an apartment in SF

Note 7/8/2013: Craigslist makes it impossible (please let me know if you were able to get around it) to allow Mechanize to retrieve HTML from Heroku’s server. However, it works when I send a request from my local server. If anyone knows why it would make a difference, please leave a comment!

———————

Anyone familiar with the SF apartment market understands the tremendous pain and frustrations in finding an available lease at a reasonable price. We have all heard and shared war stories of the hourly refreshing of Craigslist and bringing all of your bank statements and necessary paperwork to the open house just to win a chance to secure a lease for an apartment. In true SF-technology fashion, I decided to minimize response time to a new choice apartment listings on Craigslist by wiring together IFTTT, Mandrill, and Twilio to auto respond with an open email and a click-to-call to my phone number and the number in the listing to schedule an appointment.

IFTTT

IFTTT (‘If this, then that’) allows you to create ‘recipes’ using the simple conditional structure ‘If this, then that’ and various ‘ingredients‘ (i.e. craigslist queries, instagram posts, buzzfeed articles, tweets). It is powerful due to its simplicity and flexibility; it can enable anyone to put the web to work without any programming knowledge.

I used IFTTT to send an email to my email address when there is a new Craigslist query that matches the me and my roommates’ apartment listing preferences. This way, any new Craigslist listing that fits the query that I have set will be sent directly to my email inbox.

Mandrill

I setup a subdomain (I added ‘to’ as the MX record and have it setup on my Mandrill account) responsible to receiving email messages and use Mandrill to parse the inbound email (similar to how I built the A-List inbound parser). In the below code sample, it is important to note that you must first return a ‘200’ to let Mandrill know that it is the right address to send the POST request of the inbound email.

Once the email is successfully received and parsed, I use the Mechanize library to navigate to the Craigslist listing (line 31). Then, a few regex commands to find and scrape an email address (line 34) and a phone number (line 35). With the email address, I use ActionMailer (line 46) to send out a templatized email, cc’ing my roommates, asking to schedule an appointment to check out the apartment.

Twilio

If I was successfully able to scrape a phone number from the Craigslist listing, then Twilio will automatically make two outbound phone calls (one to my mobile phone and one to the number on the listing). Throughout the day, I will be able to pick up my phone at my leisure and speak with somebody (if the call connects on the other end) to schedule an appointment.

Note that the routing must also be setup in your Rails app (the sample code is not shown here).

The first step is that we are sending a request to Twilio to initiate a phone call to my phone number (agent_number or ‘555-555-5555’ in the example below) with dynamic TwiML (Twilio’s XML-like proprietary markup language).

The first dynamic TwiML (line 38 above and ‘clicktocallcallscreen.xml.builder’ below) tells Twilio to listen, upon a call connecting with my phone (agent_number or ‘555-555-5555’), for a keypad response indicating that I want to be connected with the Craigslist lister’s phone number. If I hit a key, then proceed to dial the other number and connect us. Note that I am  providing yet another dynamic action URL (‘@post_to’) that will populate the final TwiML with the correct phone numbers that will be connected in the final call.

It is in line 46 in inbox_controller.rb where the clicktocall.xml.builder (below) is called dynamically (with ‘@calling_to’ and ‘@caller_id’ both being passed to the TwiML).

The result is Twilio automatically connecting me with the phone number listed on the Craigslist post. This is a derivation of a very popular Twilio use ‘Click-to-call’ that many lead management and sales teams use to better engage with their customers.

Conclusion

Now I don’t have to be on Craigslist 24/7 to find a suitable apartment listing; I can just wait for my web server to send out emails or phone calls to schedule appointments with Craigslist posts that fit our preferences. Now if there were only a way to automate the showing up, preparing the paper work, competing against other bidders, and the remainder of the apartment leasing process.

Andy

Advertisements
Tagged , , , , , , , , , ,

Phantachat: node.js, websockets, and ephemeral conversation

In person conversation is fleeting. Words hang in the air for a brief moment then vanish suddenly. All participating members have to be present and aware in order to keep up with the conversation.

Conversation online has traditionally been very asynchronous. Instant message and email both log your messages with a timestamp, allowing you to maintain a history conversations for you to browse afterwards. You can message your friend knowing that she probably won’t respond immediately.

Phantachat is an online chat web app, but tries to mimic chat in person. Your messages linger for no longer than a few seconds, forcing you to have that page open on your screen to be able to follow the conversation. Messages are also sent in immediate real time, without having you to hit enter, and there is no backspace.

Websockets

Websocket is a web technology providing simultaneous two-way communications channels over a single TCP connection. This allows for immediate, seamless, synchronous data passing between the client and the server. We decided that in order to get the chat to behave as real time as possible, that we would choose to use Websockets (we went with Socket.io with Node.js/Express.js).

Unfortunately, there are very few free web/application hosting services out there that support websockets. Heroku doesn’t support web sockets, but you can configure your application to instead use long polling, which allows the client to send HTTP requests to the server at regular intervals and immediately receives a response.

Some other hosting platforms out there (www.appfog.com, http://www.dotcloud.com, http://www.nodejitsu.com) may support web sockets, but unfortunately they don’t provide a free tier. Our next plans are to spin up an instance on AWS so we can deploy a solution that uses websockets instead of long polling.

Dynamic chat rooms

In order to generate a unique chatroom, we created added some logic to the routing: if someone goes to the root, then generate a new hash (I used the hashids package for generating codes) and redirect to the new route with the hash at the end and start a new socket.io namespace; if someone goes to a hash that exists, then throw that person into the same socket.io namespace; and if someone goes to a route that is a hash that doesn’t exist, the server will return an error page.

With socket.io, there are two main ways of creating ‘chatrooms’: using ‘rooms’ or using ‘namespacing’. We decided to use namespacing, because namespaces can be connected to by the client (but only if it already exists on the server). With rooms, they can only be joined on the server side.

When a new namespace is created (when a new hash is generated on the server), the server creates a new namespace, then passes the hash to the client to allow the client to join that specific namespace.

Next Steps

This is still a very simple app. Next steps could be sharing other types of media that will also vanish within a few seconds (images, for instance). Or to allow more than two people to join each chat room (right now, it is only one on one).

Suggestions, feedback, and new ideas are always welcome!

Github: https://github.com/lambtron/phantachat

Andy

Tagged , , , , , , , , , ,

TextMe: A bookmarklet that lets you send highlighted text in your browser to your phone

A few weeks ago, I built ‘TextMe‘, a derpy little bookmarklet that lets you send highlighted text in your browser to your phone as an SMS. My friend Jen actually inspired this app and I realized how useful it would be when I’m on my way out and don’t want to retype the address of the restaurant into my phone. I also felt that this would be a good project to improve my javascript (I’ve never written a bookmarklet), as well as learn to work around Cross-Origin Resource Sharing.

The technology stack for this project is HTML/css/javascript/jQuery, Rails, Ruby, Heroku.

Using TextMe!

TextMe! is straightforward to use: type in your phone number in the field and generate a link that is then dragged to your toolbar. Whenever you want to text yourself something, just highlight it in the browser and click the bookmarklet.

Things I’ve learned

1. Bookmarklet Javascript: The bookmarklet, essentially, is a function that is executed whenever you click on it. In this case, the function did the following things: attach a script to the HTML document body, execute that appended script, import two functions (“getSelText” that will capture the highlighted text and “sendSMS” that will send a POST request to my server, thus sending a request to Twilio to deliver the SMS), and then calling “sendSMS” that will initiate the SMS delivery.

Here is the code for the TextMe! landing page ( www.andyjiang.com/textme ). This essentially shows how I took the user’s phone number, generated a script (with the particular phone number) that will make the sendSMS call to the server, and put that into an anchor tag that the user can drag to the toolbar. Line 17 is the javascript function that ends up in the bookmarklet that is dragged to the toolbar:

When you click on the bookmarklet after it is dragged on your toolbar, the function appends some javascript to the HTML document and imports the below two functions. Then, once everything is imported successfully, sendSMS is called with the phone number passed as an argument. The function sendSMS will get the selected text and send a request to the server (passing the recipient’s phone number and the body of the SMS, which is the highlighted text) to then ultimately send the SMS.

2. Javascript without jQuery: Because this is a bookmarklet, the intention was to keep it as lightweight with minimal dependencies. As such, it made sense (and was a great challenge) to write all of the bookmarklet javascript without jQuery to help append scripts and traverse the document object model. Note the substantial amount of jQuery used in the first code sample.

3. Cross-Origin Resource Sharing (CORS): In computing, an important security concept for browser-side programming was what is known as ‘same origin’ policy. This policy allows scripts running on the same ‘origin’ (domain, hostname, web server) to access each other’s methods, properties, and attributes with no specific restrictions. However, this policy prevents scripts on one domain to access the methods, properties, and attributes of another domain (imagine if you could write some javascript on my hostname that can call methods on another web server—there are obvious security implications and risks involved).

In some instances, it is necessary to pass data from the client side of one hostname to the server of another domain, as is in the case of this bookmarklet. TextMe! attaches and runs a script on any website on the Internet, which takes the highlighted text and POSTs that data to my humble little WEBricks Heroku-hosted web server. In these cases, CORS is one modern way of allowing for these interactions to happen.

CORS defines a specific way to allow for cross-domain requests to occur. The browser making the cross-domain request to the server has to pass an ‘Origin HTTP header’, indicating its domain. Then, the web server responds with an ‘Access-Control-Allow-Origin’ header with a value denoting which origin sites are allowed.

Please see the example from my server side Ruby code below. Line 33 an onward shows the header response with ‘Access-Control-Allow-Origin: *’, meaning to allow every cross-domain request.

Note that I didn’t include any phone number validation, which should probably be included to some extent.

Conclusion

For a quick project, TextMe! certainly challenged me in new areas and helped me learn as a developer. Understanding how bookmarklets worked, writing jQuery-free javascript, and getting a better grasp of Cross-Origin Resource Sharing proved to be achievable, yet challenging goals.

Features to add in next version:
– easier to drag the bookmarklet to the toolbar
– form validation for the phone number!

So try it out and let me know what you think!

Andy

Thanks to Jen for the original idea, Yuning for proofreading a draft of this in 30 seconds, as well as Mike and Song who helped out with various engineering aspects of the project.

Tagged , , , ,