A Look at Web Service API Design

Lately I've been spending some time working with the open source test tracking tool testlink For tracking manual testing efforts and linking tests to software requirements and other SDLC essentials it is a very nice tool. However its API leaves something to be desired. So today's topic is designing a usable API.

The purpose of the API is to allow automated testing to be tracked and reported in testlink much like manual tests are. In addition it could be used to provide a hook into an automated build system so testers are aware of new work asap. Testlink tracks test sets in two ways. First is test suites. These represent your entire pool of available tests. Second is test plans. Test plans represent some set of tests you need executed by your testers or your automation system and are taken from the test suites. Test plans are executed against builds of your software on one or more platforms (eg windows and Mac)

So with that background we can come up with some ideas for what we need in an API and compare with what testlink actually provides. Let's assume that you already have a working automated build and test system but need better reporting because there is too much data to work with without a tool to help. Since this system is already running and more or less fully automated it's pretty unlikely that anyone would be willing to manually add anything major to the database such as test case information and such.

Given that... Here's what I would expect a reasonable API to provide.

• Create a test case within a suite which may be part of another suite. Also create the suites if they don't exist.
• Create a test plan containing a list of pre-created test cases. Update as appropriate.
• Assign results to tests in a test plan run against a build for a platform.
• Create a build and associate it with a test plan (or several)

Not too much eh? The nice thing is that this hides a great deal of complexity which is handled manually in normal testing. Hiding complexity is always a major goal for a good API. Another advantage is that this concept keeps requests larger and less frequent. When dealing with distributed systems you pay a fixed time cost for every request made. That cost goes into negotiating connections, security, protocol headers and so on. So, fewer requests to a web API is usually faster.

Sadly testlink makes a very easy mistake in its API design. The developer chose to expose every simple step used in creating test plans and recording results except for a few cases where the API is more difficult to use than necessary. The list of actions look something like this

• Create a single test suite optionally belonging to another (bad stuff happens if the suite already exists)
• Create a test case belonging to a previously created test suite (ditto)
• Create a test plan
• Create a build
• Create a platform
• Assign a platform to a test plan
• Assign a test case to a test plan
• Record a result for a test case in a test plan run against a build for a platform
• Yadda Yadda blah blah.

So. A user of this API needs to jump through the same hoops as they would when doing things manually. While this saves mental effort in designing the API you end up paying for it every time you have to write code to check that a suite (and the suites it's contained in) exists and create everything that's missing via a dozen or so function calls. Not fun.

Think of an API as a time investment. The more menial bullshit your API does for you the less you have to do when using it.

One last example. A test case in testlink has a number of properties such as its author, a description and the steps that are taken to carry out the test. When you use the testlink API you provide a key (a bunch of gibbrish) to get access. The key is unique to you. Now when calling an API to create a test case using your key the API ought to be smart enough to assume you are the author unless told otherwise. The version of testlink I'm using fails to make this leap. This means you need to modify the API (I did this, small win for Open Source) or have both a user name and their key to use the API for just this one function. Again if an API can reasonably do something for a user it should.

Making of an Arcade Frontend - Part 3

Last part of this series I talked about how I started coding up the arcade frontend based on translating the features I wanted into tasks the program has to do. One thing I want to do when starting any project is work on tasks that provide the most benefit for the least effort. The strategy I used in the previous part was to pick a task that was needed for a lot of features to work (finding information on games to launch.) The final result is a nice skeleton which I can build the launcher around.

Today I'm going to talk about a second method of getting a lot of visible bang for your programming buck. That is by finishing tasks for the most visible features in the program. In short, make the user interface before other stuff. The first method (finding common tasks and doing them first) is roughly the same as bottom up development for those into jargon. This method of building programs is normally known as top-down development.

If you don't get the names try to imagine a program being like a cake. Top-down means starting with the icing and decorations while bottom-up means starting with... well... with the bottom. Both methods have their gotchas. building from the bottom up tends to mean you have something unappetizing until you're all done your work. But, building from the top down means you have a hollow shell made of icing and fondant. It looks complete and appetizing but its missing everything that makes it a cake.

With that in mind I prefer to do both at once. This way of doing things doesn't translate very well into my cake analogy. The closest idea is to build a cake one slice at a time. What you get is kinda tasty, but obviously incomplete. You end up with something that can be released very quickly that gradually improves until you're done.

Anyway; enough about cake. Let's talk about the arcade frontend... I'm going to create a graphical menu to pick out a game to start. I drew inspiration mainly from the rotating song menu in Dance Dance Revolution. Playable games will show in a semicircle with the selected game in the middle at a larger size than the others. selecting a new game rotates the menu and you can rotate forever in either direction.

To get the right look requires a bit of math. placing the titles vertically is simple... sort of. Dividing the screen's height by the number of visible items will give you the amount of space each item can take. If you take that number and halve it you get the distance to the center of each space that can be used.

Now, I'm guessing a few people are asking why I need the center of the space where each item goes at all since PyGame draws images positioned at their top left. The reason is that the center of each item will stay the same no matter how large or small the item gets. It's very easy to calculate the top left of the image by subtracting half of the image's size from its center. It's not so easy to calculate the top left of the image relative to the top left of the  The image below shows this better than words can.

Another thing to note is that I'm using an odd number of games in the visible part of the menu. This way there's one "middle" game which is the selected game and an even number of other games around it. Having an even number of games would result in the selected game being off center which looks ugly.

Placing the titles horizontally requires a bit more math knowhow but not much. If you remember your trig class you might recall that the sine function looks like a smooth set of waves like this.

Source: http://images.yourdictionary.com/sine-curve

In programming the sine function usually uses a unit called radians. Radians are how mathheads measure angles. You can do some cool tricks with them once you get it. The important part is that half a circle (180 degrees) is equal to Pi (3.14159...) radians. The cool part is that sine between 0 and Pi radians looks exactly like half a circle. In fact the value of sine at any angle matches how far you are from the center of a circle vertically at that angle.

With that all that needs to be done is to multiply the item number by Pi then divide it by the number of items that are to be visible plus two, put that number into a sine function, then multiply the result by the radius of the circle. Why plus two? In order to support spinning between games an extra game is to appear offscreen and slide onto the screen when changing selection. If we don't take these two items into account games coming onscreen from the top or bottom will wobble to the left or right.

Here's the result of all that math

Not too bad eh?

Next entry I'll be showing some code and talking about how to handle transitions between titles. I'll also show you the code that this entry produced as well.

PONIES! And random shite

Yes. I admit it. I'm a fan of My Little Pony: Friendship is Magic. After finding a mysterious new trend of pony image macros with a relatively modern art style I got curious. A brief bit of google-fu revealed the show so, out of curiosity, I watched the first episode... then the second, and third, and fourth... and, yeah.

So here's some random clips I find amusing.

http://www.youtube.com/watch?v=OPADMGqN1Uk&feature=related

http://www.youtube.com/watch?v=h7PGKcikaGY&feature=related

http://www.youtube.com/watch?v=hWD2Fb4Pneo&feature=related

http://www.youtube.com/watch?v=FbXh7ZipUuQ

http://www.youtube.com/watch?v=1A6NxMV4Lqo&feature=related

And some non-pony bits... just for fun

http://www.youtube.com/watch?v=RISJ--pvxmU&feature=related

http://www.youtube.com/watch?feature=endscreen&NR=1&v=jluv2HxFEqs

Making of an Arcade Frontend - Part 2

If you haven't read them yet check out the prolog and Part 1

I think it's been a sufficient ahem an excessive time since I've posted last...

During my Christmas vacation I managed to bash out the majority of the features I've wanted to support in my arcade frontend. I had a couple false starts before then. Frustrated, I went with the tried and true "just make the damn thing work" development methodology (aka hacking.)

Well, I didn't build it quite as haphazardly as that. The idea is to translate the project requirements into some minimal functionality that must be created for the project to be considered "working." For a small project like this it only takes about 10-30 minutes of effort. For larger projects it takes longer. Most of the time you'll find some features that are independent of each-other. To get the most bang for your buck your best bet is to start working on some feature that a lot of requirements depend on.

Huh? What?

I'll give an example. This is taken from the requirements list in Part 1 under each requirement is a number of functions that need to be implemented

• Hide the regular Windows front-end.
• Using pygame run in full screen mode
• Leave full screen mode before starting a game, re-enter once the game exits
• Allow the user to shut down the computer without exiting to Windows
• Method for responding to user input
• Method for confirming user action
• Method for invoking windows shutdown
• Allow an administrator to exit to windows
• Method for responding to user input (key-mapped arcade controls)
• Clean program shutdown
• Allow rapid selection and launching of games using arcade controls (not keyboard and mouse)
• Method for responding to user input (key-mapped arcade controls)
• Support launching any game/program, not just MAME
• Scan (something) to create a list of programs that are launchable
• When prompted launch a program using information from the list
• Display a preview of the selected game
• Scan (something) to create a list of images associated with games
• Method to load images as needed 
• Display a list of games that can be played
• Scan (something) to create the list of playable games
• Method for distinguishing selected game from others
• Method for displaying available games 
• Display a marquee for the game... somewhere near the screenshot
• Scan (something) to create a list of images associated with games
• Method to load images as needed
• Everything must have an animation
• This means needing an endless update/draw loop for objects
• Need a state machine to indicate whether some transition is occurring and what kind
• Alter control-key mappings to support games without configurable controls 
• Method to invoke IPAC-2 programmer with some known configuration
• Scan (something) to create a list of control configs associated with games  

There are a couple things that pop out:

• Method to load images as needed
• Scan (something) to create a list of games and things associated with them

Of those scanning comes up the most and, more importantly, relates most directly to the purpose of the program which is starting games.

For my first crack at this I've decided that (something) will be a directory tree containing files called (game)-launcher.cfg which describe playable games. For the sake of organization, since there will likely be well over 100 files, the config files can be in any directory in the tree.

What do I want to put in the file:

• The program to start
• Extra information to pass to the program
• Controller settings for the game
• Where to find game's marquee, screenshot and other image files

Now, how do I format the file? I don't want to spend an undue amount of time writing a parser so I could use one of Python's built in parsers. But that still means I need code to convert that to an easily usable datastructure (let's just ignore JSON for now since it slipped my mind at the time) So... how about just making it a Python module. That reduces parsing to one line of code. The file itself simply contains a Python dict with a standardized name and standardized keys.

First the code to scan for config files:

def FindAllLaunchables( startDir ):
    outList = []
    dirList = os.listdir( startDir )
    for dir in dirList:
        checkDir = os.path.join( startDir, dir )
        try:
            if os.path.isdir( checkDir ):
                outList.extend( FindAllLaunchables( checkDir ) )
            elif os.path.isfile( checkDir ) and dir.lower().endswith("-launcher.cfg"):
                outList.append( checkDir )
        except WindowsError:
            pass
    
    return outList

That's it. The function recursively scans all directories under startDir and, if it finds a configuration file adds its location to an output list which the function returns.

This bit of code handles reading the configuration file.

def CreateLaunchers( configFiles ):
    launchers = []
    for launcherConfig in configFiles:
        moduleName = os.path.splitext( launcherConfig )[0].replace( os.path.sep, "_" ).replace( ".", "" ).replace( " ", "_" )
        
        moduleFile = open( launcherConfig )
        try:
            modulePath = os.path.split( launcherConfig )
            module = imp.load_module( moduleName, moduleFile, launcherConfig, ["cfg","rt",imp.PY_SOURCE] )
            launchers.append( config.Launcher( module.LauncherConfig ) )
        finally:
            moduleFile.close()
            del moduleFile
    return launchers

The meat here is the call to imp.load_module. The imp module exposes some of the guts of Python's import statement and is useful for importing arbitrary files as modules. Notice the config.Launcher object instantiation. The Launcher class will be used to carry out the work of starting up a game based on its configuration along with some other stuff.

And that's everything! To finish this initial step I stubbed in some code to start up fullscreen mode and to pick up keyboard input. That code is below and makes up the skeleton which the rest of the program will be built around.

pygame.init()

# load configuration
configFiles = FindAllLaunchables( search, skip )
launchers = CreateLaunchers( configFiles )

# startup sequence
screen = SetScreenMode()
pygame.key.set_repeat( 1000, 200 )
pygame.mixer.init()
things = []

# the main loop!
frameStart = time.clock()
time.sleep( 0.01 ) # XXX: don't want a frame time of 0 when starting
while True:
    frameTime = time.clock() - frameStart
    frameStart = time.clock()
        
    for event in pygame.event.get():
        if event.type == pygame.QUIT or (event.type == KEYDOWN and event.key == pygame.K_F12):
            sys.exit()
        
    for thing in things:
        thing.Update( frameTime )
        thing.Draw( screen )
    pygame.display.flip()
    screen.fill( (0,0,0) )

Whelp, that's all for part 2. I'll have more for you next time! (and since the project is working, next time should be sooner, not later) SetScreenMode isn't all that exciting so I've spared you its details. Notice that with this code we now have the following done.

• Using pygame run in full screen mode
• An endless loop to update/draw objects
• Method for responding to user input (stub)
• Scan (something) to create a list of games and things associated with them

Not bad for one blog-entry worth of work eh?

WOO!

Or more precicely Sandra and Woo

Oddly enough I found this comic through a banner ad posted on Something Awful's LP archive. I was reading through the epic that is Boatmurdered (no graphic violence, unless you count a white ASCII smiley turning into a red ASCII smiley as violent) I wasn't entirely sure what to expect but the ad interested me so I clicked. And have lost many hours reading through a highly approachable, yet surprisingly mature comic strip. Amusingly, the ad is still running now.

Initially it's something like Calvin and Hobbes... yet it's nothing like Calvin and Hobbes at the same time. The characters are all older (Sandra's 11, at least at first) and more mature. There's also more budding romance. In any case you should read it. The strip relies a great deal on characterization so some strips don't make much sense unless you know the characters but there's still plenty of strips which stand alone nicely.

Oh! And I almost forgot! There's a new Hawken trailer and more gameplay footage. I am itching to play this game so badly!

Another Gem

Posting this from work since I'm doing some boring data uploading crud.

If you ever find yourself doing this as a programmer/software engineer, please reconsider (your career choice)

From: http://download.oracle.com/docs/cd/B10501_01/appdev.920/a96624/03_types.htm#29460
Note: Currently, VARCHAR is synonymous with VARCHAR2. However, in future releases of PL/SQL, to accommodate emerging SQL standards, VARCHAR might become a separate datatype with different comparison semantics. So, it is a good idea to use VARCHAR2 rather than VARCHAR.

Brilliant! Create a new type because maybe at some point you might want to change the behavior of an existing type at some unspecified time in a vaguely defined future. Apparently the guys at Oracle decided it made less sense to introduce a new type when the need arises (what sane people do) instead of adding a redundant type right away... just in case. Oh! And make the new type have the old behavior just to conveniently screw up backward compatibility if the original type does change. That's like making "Toy Story" the sequel to "Toy Story 2"... WTF

Fiftith!

So, what could be more celebratory than explosions and shooting? Nothing! That's what

http://www.youtube.com/watch?v=5HpaWcDX0f4

http://www.youtube.com/watch?v=UvvQRIxXGzA&feature=fvsr

http://www.youtube.com/watch?v=nY6nm-6eCzM&feature=related

http://www.youtube.com/watch?v=9La3Yc2u3NM&feature=related

http://www.youtube.com/watch?v=aOx_wHhitqk&feature=related

http://www.youtube.com/watch?v=fOJppJSKG9Y

And last but not least. The loading and firing of the main guns of the Iowa class battleship.
http://www.youtube.com/watch?v=BJdinQQk1O8&feature=related