My First Instructables Post!

If you are skimming, my instructables pages can be found here.

If you want a small explanation and summary of the project, please, continue........  :)


For my Tangible Interactive Computing Class (838F) our final project had to include, among other things, an Instructables page on the site www.instructables.com.  The creation of this page was to replace the typical research paper that usually accompanies a term project.  The idea was to force us to create a step-by-step document of how we built our projects.  The recent explosion of open source hardware/DIY makers is fostered by such websites and blogs where people share their experiences working with hardware such as the Arduino.  

Our project (myself and Preeti Bhargava) was, to put it simply, a personal informatics black box that you clip onto your belt.  As you go about your day your activity, surrounding environment and locations are tracked to produce a report telling you how much time you spent inside and outside.  The device leverages off of a system-of-systems providing context-aware analysis for such ubiquitous computing devices.  We make sense the raw data signals from the multitude of sensors through our back-end system that uses many external API calls, home-grown context-aware systems and AI decision tree analysis. 

A brief video explanation of our project:

We in the class all agreed it would be a great idea to post something to the site vs. a research paper, but I believe we were all surprised at how receptive and supporting the community is!  Four of of the seven projects (you can view all seven here) were eventually featured on the front page of the site!  Our page currently has over 20,000 views!

So, if you haven't checked out our step-by-step process yet, please do!  It shows how the hardware was made, including the soldering involved, the 3D-printing and end product.  I hope you are somewhat inspired or excited to explore this area of DIY hardware hacking.  I know I feel empowered as a result of the projects that I contributed to in this semester-long class.  As a CS Master's/PHD student I typically took the hardware of the systems I was working with at face value.  I now know I can not only create the software that runs and extracts data from hardware but also design my own hardware and sensor systems!

I thank Dr. Jon Froehlich for that hardware liberation!

Webservice to determine if a Latitude / Longitude Point is inside a polygon or building

I spent some time trying to find an easy, do-it-yourself method to build a web-service that would take a Lat/Long as input and tell me if that point is contained within a polygon.  For my purposes I wanted to know if a point was within a building or not and a polygon will easily satisfy this request.

The Ray-casting algorithm is a method to determine just that.  Google maps and Esri have function calls that one can use within their API's, but those are client-side as you can only call such functions through javascript, JQuery or the like.  Thus if you want something on the server side you must write it yourself.

Luckily I found this rosettacode side that had the ray casting algorithm coded in several languages!  Their example uses pre-canned polygons and points.  I modified it to accept a lat/long from a user and return true if the point is within any of my polygons/buildings and false otherwise. The list of polys contain the Lat/Long points for each building.



My python code:

# Code was taken from http://rosettacode.org/wiki/Ray-casting_algorithm#Python
# uses simple ray-casting algoritm to see if point lies within a polygon
# 
# Adapted by Nick Gramsky to be used as a service to check if LatLong was inside  
#    a GIS polygon
from collections import namedtuple
from pprint import pprint as pp
import sys
 
Pt = namedtuple('Pt', 'x, y')               # Point
Edge = namedtuple('Edge', 'a, b')           # Polygon edge from a to b
Poly = namedtuple('Poly', 'name, edges')    # Polygon
 
_eps = 0.00001
_huge = sys.float_info.max
_tiny = sys.float_info.min
 
def rayintersectseg(p, edge):
    ''' takes a point p=Pt() and an edge of two endpoints a,b=Pt() of a line segment returns boolean
    '''
    a,b = edge
    if a.y > b.y:
        a,b = b,a
    if p.y == a.y or p.y == b.y:
        p = Pt(p.x, p.y + _eps)
 
    intersect = False
 
    if (p.y > b.y or p.y < a.y) or (
        p.x > max(a.x, b.x)):
        return False
 
    if p.x < min(a.x, b.x):
        intersect = True
    else:
        if abs(a.x - b.x) > _tiny:
            m_red = (b.y - a.y) / float(b.x - a.x)
        else:
            m_red = _huge
        if abs(a.x - p.x) > _tiny:
            m_blue = (p.y - a.y) / float(p.x - a.x)
        else:
            m_blue = _huge
        intersect = m_blue >= m_red
    return intersect
 
def _odd(x): return x%2 == 1
 
def ispointinside(p, poly):
    ln = len(poly)
    return _odd(sum(rayintersectseg(p, edge)
                    for edge in poly.edges ))
 
def polypp(poly):
    print "\n  Polygon(name='%s', edges=(" % poly.name
    print '   ', ',\n    '.join(str(e) for e in poly.edges) + '\n    ))'
 
if __name__ == '__main__':
    polys = [
      Poly(name='avwilliams', edges=(
        Edge(a=Pt(x=38.9913160, y=-76.937079), b=Pt(x=38.991333, y=-76.936119)),
        Edge(a=Pt(x=38.991333, y=-76.936119), b=Pt(x=38.990287, y=-76.936108)),
        Edge(a=Pt(x=38.990287, y=-76.936108), b=Pt(x=38.990278, y=-76.937057)),
        Edge(a=Pt(x=38.990278, y=-76.937057), b=Pt(x=38.990495,y=-76.937052)),
        Edge(a=Pt(x=38.990495,y=-76.937052), b=Pt(x=38.990499,y=-76.936424)),
        Edge(a=Pt(x=38.990499,y=-76.936424), b=Pt(x=38.991091,y=-76.93643)),
        Edge(a=Pt(x=38.991091,y=-76.93643), b=Pt(x=38.991104,y=-76.937079)),
        Edge(a=Pt(x=38.991104,y=-76.937079), b=Pt(x=38.9913160, y=-76.937079))
        )),
    ]  

    if len(sys.argv) != 3:
       print "Incorrect number of arguments.  Please submit a lat and a long...."

    userpoint = (Pt(x=float(sys.argv[1]), y=float(sys.argv[2])))

    testpoints = (Pt(x=float(sys.argv[1]), y=sys.argv[2]), Pt(x=38.990842, y=-76.93625),
                  Pt(x=38.9021466, y=-77), Pt(x=0, y=5),
                  Pt(x=10, y=5), Pt(x=8, y=5),
                  Pt(x=10, y=10))
 
    #print "\n TESTING WHETHER POINTS ARE WITHIN POLYGONS"
    inside = False
    for poly in polys:
        #polypp(poly)
        #print '   ', '\t'.join("%s: %s" % (p, ispointinside(p, poly))
        #                       for p in testpoints[:3])
        #if ispointinside(testpoints[1], poly):
        if ispointinside(userpoint, poly):
           inside = True

    print inside

And the corresponding php code:

Oh and this was done on a mac, hence the need to call out 'python' in the system call. Syntax for this call will vary by OS. Of course this example is using a hard-coded polygon within the python script. That's fine if you want to check only a few polygons/buildings but not a great way to code if you want to check many buildings. One could simply create a small database that stores polygons and iterate through each polygon. Using PostGres you can perform a bounds call to retrieve only polygons that contain the point within their minimin bounding rectangle and iterate through this python code. Additionally this could very well be coded in php but I just didn't have the time :) It would reduce the need to perform a system call and reduce the software dependencies....

 <PUBLIC SERVICE ANNOUNCEMENT> Yes, my php code IS very light and simple. One should ALWAYS clean user input prior to working with the parameters they provide. My example/prototype did not include that (BAD DEVELOPER) but this was an internal prototype only. </PUBLIC SERVICE ANNOUNCEMENT>

How I gained over 100 Twitter followers in one week without writing one Tweet. EVER....

So I stumbled on this by accident a little while ago while working on TwitterStand earlier this semester.  I thought it was quite interesting and worth a (semi-)formal study.  So I set up a new Twitter account @UniqueShout.

This is what the account looked like prior to adding any new followers:

Just your standard, normal account that follows 5 accounts and sits idle.  I created the account on May 25.

Almost instantly I followed 1000 people using the Twitter API.  36 hours later I followed another 790, however I was already being followed by 30 Twitter users at this point in time...

One week later I'm sitting at 105 followers, yet I still yet to produce A SINGLE TWEET!

It's now been two weeks and the rate of new followers has subsided, yet I do currently have 111 followers.  There have been several accounts that stopped following me, yet I believe these are spam/porn accounts.

So you're likely saying "Most of these accounts must be spam/porn accounts!".  Well, not exactly.  Porn seems to account for 10% of the followers.  I think many of the accounts that dropped were spam accounts.  Let me first step back and say something about the accounts in general that are following @UniqueShout.  They fall into one of four categories:

1. Spam/porn accounts - These accounts are not real people but the exhibit the following behavior:
1. All have a hyperlink in their 'about' section that leads to (what I believe to be) a porn site.
2. They all produce 3 Tweets over a 36-hour period.  Each Tweets looks to be a crafted Tweet that is about nothing yet has text to possibly fool spam filters like those found on email servers.
3. All accounts produce their Tweets at roughly the time, suggestive these accounts are run by the same person/entity.
2. Real people who love to Tweet.  These are people who Tweet about their daily lives and random thoughts.  This looks to be 10% of the people who are following this account.
3. Legit newsworthy Twitter users, like local CNN corespondents, local news affiliates and other micro bloggers.  This looks to be ~30% of the followers.
4. Mixed accounts that include small businesses, organizations and users that communicate over Twitter.  I don't consider these personal accounts as the account might be a front for a small business, a personal business or organization.  This appears to be about 50% of @UniqueShout's followers.

Here is the latest snapshot from the account showing 111 followers.  Note some of the accounts that are following @UniqueShout, one of which is even @ESPNDeportesMia

I think the following take aways should be noted here:

1. It's quite interesting WHY the account has so many followers despite the lack of Tweets.  I'm now curious about the possible mining techniques Twitter-mining sites have to decide who to follow.
2. I may have produced a method for identifying spam.  While not optimal or exhaustive, it appears to find accounts that are owned by similar entities.  Is this a possible method to trap/ID spam BEFORE it does its dirty work?
3. The accounts I followed are mostly news-spreading accounts.  But not all of them.  2% of the accounts have never Tweeted and are followed by very few people.  Some of the accounts are icons on Twitter (@cnn for example) yet some are small local news outlets.  A further study should find the minimal subset that produces the most followers.
4. I'm unsure what happens when I stop following the accounts @UniqueShout is following.  This will be a follow-on study.
5. I'm unsure if this is a good method to add followers or not.  One thing is for sure, if you want a high-follower count, this is a way to get a decent amount quickly and easily.  So if you're setting up a small business for example, this might be a method to jump-start your Twitter account.  At some point I'll do this to my established, personal account and see if I get similar results.

I'll take a deep dive into these accounts in my next blog.