David Lee's Blog

Retiring yahoo fantasy greasemonkey scripts

2009-10-27T22:27:00.000-07:00

Two years ago, I wrote a couple of popular greasemonkey scripts that allowed yahoo fantasy users to easily see live stats for their fantasy team. At the time, users who didn't shell out the 10 bucks to buy stattracker had to wait until the following day to see how their players performed. I didn't think that was fair so I automated the process of clicking through the boxscore link, parsing out the stat for a player, and summarizing it in a nice modal window. Starting this football season and continuing with basketball today, yahoo has decided to offer stattracker for free! This means that the scripts I released are no longer needed. Competition from other fantasy sites most likely compelled yahoo to offer it for free, but I'd like that all of you who used it, told your friends about it, and sent me bug and suggestions also forced their hand :)

initializing js objects with many parameter

2008-09-04T23:40:00.000-07:00

For the past couple of months I've been working on a fantasy football application on facebook. I worked primarily on the live online draft and real time fantasy scoring.

It was a lot of work and also a lot of fun. My coworkers and I spent many a night and weekend getting the site looking and working correctly. I was worried that I would get sick of thinking about football but luckily that never happened. I want to share what I learned on this project so I thought I'd start with an easy one that is now part of my personal pattern repository (ppr).

What's the best way to initialize a js object with lots of initial state? Originally, I passed in function parameters: function init(param1, param2, param3). As the complexity increased, I found myself passing in more and more parameters.

the init string was generated on the server side so my code looked like
String.format("init(%s,%s,%s,%s);", param1, param2, param3, param4).

Whenever I added a new parameter I had to change the server side code and also the js file.

After a couple of changes I wizened up and found a much better way to do things. on the server side, I created a JSONObject and set up my parameters like so:

JSONObject json = new JSONObject().put("param1", param1).put("param2", param2)....
String.format("init(%s)", json.toString())

And on the js file:

function init(params) {
var param1 = params.param1;
var param2 = params.param2;
...
}

Your 2008 San Francisco Giants

2008-03-31T21:29:00.000-07:00

Goodbye Barry Bonds, hello Brian Bocock. This year's team is going to be tough to watch. The heart of our lineup is Randy Winn, Bengie Molina, and Ray Durham. I expect Aaron Rowand to move up from his sixth spot in the lineup within a few games but as of opening day, the 3-4-5 hitters are embarrassing. Thankfully, baseball is not all about offense and the strength of this team is the starting pitching. Pundits predict that we'll win around 60 games this year but I really think we have a good shot at .500 if the pitching performs as expected and if our hitters exceed expectations. Without further adieu, here are my optimistic predictions for the 2008 season.

Pitching

The trio of Matt Cain, Tim Lincecum, and Barry Zito will win 45 games.

The Giants bullpen will be the strength of the team and will be ranked top 5 in the NL in holds, ERA, and WHIP.

Hitting

Aaron Rowand will hit above .300, steal 15 bases, and drive in 100 runs.

Rajai Davis and Eugenio Velez will steal a combined 60 bases.

Benjie Molina will hit .290 with 30 homers.

sorry...I can't do this anymore. Here's my tip for all Giants fans this season: Don't miss Cain and Lincecum pitch and check out the 2008 promotional schedule for some bobbleheads.

2007 Fantasy basketball script released

2007-11-04T00:06:00.000-07:00

*update - 2/22/2008 - updated the script to handle new yahoo boxscores for in progress games

I couldn't sleep last night (Friday night...actually Sat at 2am to be exact) so I stayed up and implemented the two missing features for the live stats basketball script: maintaining the order of players in the modal window and displaying a summary of all active players' stats.

The first feature was easy to implement since I had already solved that problem for baseball. The old technique of adding a row after retrieving a player's stats leads to indeterminate ordering because we do not wait for a boxscore request to finish before moving on to the next. Therefore, it was possible that a UTIL player who is towards the bottom of the management page would actually be processed faster than a PG depending on how snappy the server handled the request. The solution is to create placeholder rows that have the correct order and then update those rows later.

The second feature is summarizing all your players' stats for the night. Unlike a baseball boxscore where a player's stats appear on multiple parts of the page and some stats need to be derived, all basketball stats appear on one row. That's why I didn't create full blown function objects that had getter/setters for stats like assists, turnovers, pts, boards, etc. Instead, I went the lightweight route and defined two rules that are used to summarize stats. If the stat has a dash, like 7-11, then we keep track of total made and total attempts, else we just sum up the current stat with the running total.

halloween treat

2007-10-31T23:00:00.000-07:00

No candy to pass out but I did make a few tweaks to my fantasy basketball live scoring script. Like the modified baseball version, the script no longer automatically loads. The user has to trigger it by clicking on the 'show freebie stats' link. I also ripped out the autocenter on scroll code since that was causing display issues for users on low resolution monitors.

The two biggest missing features that I will tackle next are maintaining the order on the display and totaling all your active players' stats. But I probably won't get to that until the weekend. don't fret, yahoo provides free stats for the first couple of weeks of the season so you're covered.

annoying commercial parody

2007-10-14T17:31:00.000-07:00

SNL lampooned one of the most annoying commercials that has been playing during the LCS.

Fantasy baseball recap

2007-10-02T21:37:00.000-07:00

Back in March, Yahoo Sports invited average Joes to go head-to-head against their fantasy experts in baseball. I answered the call and was pitted in a 16 team, head-to-head league with Big Noise himself, Brad Evans. My key draft picks were Chipper, Magglio, Markakis, and Jose Reyes. My waiver wire gems were Ryan Braun and Carlos Pena, who I swapped for Derek Lee just before the trading deadline. My offense carried me the entire season because my only good pitcher was Josh Beckett. Since it was a head-to-head league, I knew that if my offense held up, I could stream pitchers if I reached the playoffs.

Only 6 out of 16 teams qualified for the playoffs and I made it as a 3rd place qualifier. This meant that I didn't have a first round bye which is always a bit dangerous. Luckily, my team held up and I moved on to the semifinals against the 2nd place team. We both had off weeks but my team went on to win. Waiting for me in the championship showdown was Brad Evans' team. He was on a roll and I had to go through him to win the championship. I was up to the challenge and pulled out all the stops like streaming pitchers to win the W and K categories and picking hot September additions such as Jacoby Ellsbury and Norris Hopper.

Just like the regular season, my offense carried me and streaming pitchers yielded wins in the W and K categories. In the end, it was a fun season and I'm thankful that Yahoo opened the league up and that Brad Evans took some time to go up against some of his readers.

Pics or it didn't happen:

Padres out, Rockies in

2007-10-01T22:16:00.001-07:00

Thank goodness for TBS. Too bad this game was not televised on the networks because it was an instant classic that saw the Rockies come back against the man with the most saves in the history of baseball. I actually didn't see the comeback because I was peeved that Scott Hairston hit another clutch homer for the Padres and went to do the dishes. This guy has been killing the Giants all year and during the last week of the regular season he tossed the bat about 20 feet after blasting a homer off Matt Cain. Cain let him have it and it'll be interesting to see these guys matched up next year. When I came back from doing the dishes, I saw that the Rockies had won and that Holliday scored the winning run. What a relief that must have been since he was the one that misplayed the Brian Giles flyball that led to the Padres tying the game. If this game was any indication how the playoffs will be played, then we're all in for a treat. My sleeper pick is the Cubs but I think either Boston or Cleveland will win it all.

Converting a large AVI file to a smaller WMV file for picasa or youtube

2007-10-01T22:02:00.000-07:00

I was uploading pictures and videos from a company outing and soon discovered that the max file size for uploads to PicasaWeb is 100MB. So I decided to give YouTube a try but the max upload size on YouTube was also 100MB.

I took the video on my digital camera and it was in AVI format. I needed something that would convert this file to a smaller size. A couple of google searches later I discovered that Microsoft has a free WMV converter. The program provides different output quality settings and soon I was able to get my 162MB avi file to a 11MB wmv. Here's the download link: http://www.microsoft.com/windows/windowsmedia/forpros/encoder/default.mspx

If you aren't running Windows, mplayer might be sufficient for your video conversion needs. But the Linux version of Picasa doesn't allow uploading to web albums so you will have to do it through the web interface.

Giants 2007 Recap

2007-09-28T00:01:00.000-07:00

Now that the season is winding down, it's time to revisit my extremely optimistic outlook for the Giants 2007 baseball season. Granted, some of these predictions (see Benitez) were best case, Brady Anderson ala 1996 scenarios, but as the 2005 White Sox proved, it can happen.

Pitching

Prediction: Zito will assume ace responsibilities and that new windup may give him that extra zip and movement.

Reality: The new windup was abandoned in the first week of spring training because the Giants didn't pay for new windup Barry. Barry had one of his worst seasons as a professional.

Running tally: 0-1

Prediction: A seasoned Matt Cain, aka Bobby Hill, will have learned how to pitch

Reality: Cain's W-L record belies how he well he pitched because the record could have been reversed if the Giants gave him more run support or if the relief pitchers didn't blow his leads. He definitely matured from his rookie year and mixed sliders and change ups to go with that nasty fastball.

Running tally: 1-1

Prediction: Noah Lowry might be able to fool hitters once more

Reality: Before Noah got injured he was among the league leaders in wins. He doesn't have overpowering stuff and runs a lot of counts to 3-2 but he still keeps the games close and finds ways to get guys to miss on fastballs never topping 90.

Running tally: 2-1

Prediction: Matt Morris will be able to avoid that one big inning blowup he always has

Reality: nope. same old Morris after a promising start and eventually traded to the Pirates for Rajai Davis.

Running tally: 2-2

Prediction: The fifth starter, whether it is a renewed Russ Ortiz or rookie sensations Linceum or Sanchez will surprise the league by storm.

Reality: I sure gave myself some leeway here by counting on at least 1 of 3 players to have a good year. Lincecum really did show the league that he's a special player. Whenever he pitches the opposing team is always at the railing checking him out.

Running tally: 3-2

Prediction: Benitez will be in shape and less of a headcase.

Reality: Nope. same old Benitez. Traded before the break to the Marlins for Randy Messenger.

Running tally: 3-3

Hitting

Prediction: old guys will be injury free

Reality: wishful thinking. Durham, Roberts, Bonds spent significant time on the DL.

Running tally: 3-4

Prediction: Barry Bonds will break Hank's record by early August so the team can concentrate on the playoff push

Reality: Barry broke the record in early August but playoff hopes were pretty much gone at that point. half right = half wrong = a push

Running tally: still 3-4

Prediction: Dave Roberts will be that leadoff hitter we haven't had since Brett Butler

Reality: injured for most of the first half, came on strong for awhile and then leveled off. Splitting time with Rajai Davis so can't say he's that good.

Running tally: 3-5

Prediction: Rich Aurilia pounds 40 homers and shows Seattle fans why they once signed him to big bucks

Reality: yeah right. that was the last prediction so I went for the juggernaut. Rich Aurilia couldn't beat out Ryan Klesko for the starting first base job.

Final tally: 3-6 which pretty much sums up the Giants season. But next year, we'll be back.

Logging request and response headers in Tomcat

2007-09-26T22:10:00.000-07:00

What's the best way to dump out request or response headers in Tomcat?

Before rolling up your own solution, take a look at the valve component that Tomcat supports. In particular, the aptly named RequestDumperValve will log all the request and response headers in Tomcat. To enable the valve, just go to your server.xml file in the /conf directory and uncomment this line:

Then restart your server and the headers will appear in your log file.

Javascript composite comparator

2007-09-25T23:12:00.000-07:00

Since it's available on every modern web browser, Javascript captures the title as the world's most deployed programming language. But while its availability is almost universal, its reputation is less than stellar. Douglas Crockford wrote a great piece on Javascript being the world's most misunderstood programming language. He lists several reasons such as design errors, bad early implementations, and even naming the language Javascript which implied it being like Java and it being no more than a simple scripting language.

I'll admit that my initial opinion was quite down as well. I used it strictly as a C clone and cringed anytime I had to debug a script. But now that I have had more exposure to and experience with the language features such as closures and first class functions, I've definitely reversed my opinion. It's so useful to perform client side processing with such a full fledged language at my disposal.

This post wouldn't do Javascript justice if it didn't have some accompanying code, so I'll revisit my Java generics and varargs post and reimplement the composite comparator using Javascript.

To recap, the composite comparator (aka cascading comparator or chained comparator) is a comparator made up of several other comparators. My Java 1.5 implementation uses the varargs language feature to yield a simple to use api that accepts a variable amount of comparators.

Since javascript is dynamically typed and supports variable length arguments, the implementation looks pretty similar.


This is our player object:



function player(name, hr, rbi, sb) {

    this.name=name;

    this.hr=parseInt(hr);

    this.rbi=parseInt(rbi);

    this.sb=parseInt(sb);

}



Let's create some players and populate a players array:

var a = new player('Arod',50,150,1);

var b = new player('Reyes',50,150,2);

var c = new player('Bonds',50,150,3);



var players = new Array(a,b,c);



Next, we need to create some comparators:

function sortRbi(a, b) {

   return a.rbi - b.rbi;

}



function sortHr(a,b) {

   return a.hr-b.hr;

}



function sortSb(a,b) {

   return a.sb-b.sb;

}



Here's the composite comparator:

function compositeSort() {

  var args = Array.prototype.slice.call(arguments);

     return function(a,b) {     

     for (var i=0;i<args.length;i++) {

        var result = args[i](a,b);

        if (result!=0) { 

           return result;

        }

     }

     return 0;        

    }

}



Here's a negate comparator:

function negate(func) {

    return function(a,b) {

        var i=func(a,b);

        if (i==0)

          return i;

        else

          return -i;

    }

}





This comparator will sort by hr ascending, rbi ascending, and sb descending.

var comp = compositeSort(sortHr,sortRbi,negate(sortSb));



And now we pass the newly created comparator to the built in sort method in arrays.

players.sort(comp)

xslt stylesheet to convert xml to tab delimited file

2007-09-24T23:20:00.000-07:00

Sometimes, you just want a tab delimited text file. For those occasions, here's the xslt to convert an xml file to a tab delimited file. the tab and newline characters might get converted when displayed on the screen so do a view source if the select=" " shows whitespace

<xsl:stylesheet version='1.0'
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes"/>
    <xsl:variable name="tab" select="'	'" />
    <xsl:variable name="newline" select="'
'" />

    <xsl:template match="/">
        <xsl:apply-templates select="//player"/>
    </xsl:template>

    <xsl:template match="player">
        <xsl:value-of select="first_name"/>
        <xsl:value-of select="$tab" />
        <xsl:value-of select="last_name"/>
        <xsl:value-of select="$newline" />
    </xsl:template>
</xsl:stylesheet>

It's official. The Barry Bonds era has come to an end

2007-09-21T17:21:00.001-07:00

It's official. Barry Bonds will not be a San Francisco Giant next year.

Many people outside of the Bay Area don't understand the relationship Giants fans have with Barry. We know he took steroids. We know he's arrogant. We know he's mean to reporters. So why is it that we don't hate him like the rest of America? Because he's a Giant. Because he grew up in the Bay Area. Because his dad played for the Giants and because his godfather is Willie Mays. Because he's not the only guy that ever took steroids and because he was a hall of famer caliber player before he juiced up. Because 1993 was a magical season.

To be competing for a division title a year removed from a woeful season and rumors of a move to St. Petersbug was invigorating. Suddenly it was fun to be a Giants fan again. Barry put the franchise on his back and carried the organization through the opening of Pac Bell Park, into the World Series in 2002, and even up to when he broke the home run record. That's a lot of great memories. He's been the face of the Giants for 15 years and when you're loyal to your team, you're going to be loyal to the face of the franchise too. Go Giants!

Would you leave 63 million on the table?

2007-09-20T22:47:00.001-07:00

Andrei Kirilenko would. He's so unhappy playing for Jerry Sloan that he's willing to forego the rest of his max contract which equates to 63 million dollars. Is this guy for real? Most guys want to void or renegotiate their contract to make more money. Kirilenko's already got a max deal and the way he's played the last two years he's never going to get that sweet a deal again.

He played for the Russian team this summer and was the main man so maybe he missed being the go to guy. Maybe he just really misses home. Maybe he was misquoted and the interview was taken out of context. Maybe a Russian oil tycoon has promised him even more cash to play in Russia.

Whatever the reason I'm sure more of the story will come out in the coming weeks. I've always been a big Kirilenko fan. He won me over many years ago when he went ballistic in a Rookie vs Sophomore game during All star weekend. The other players were playing lackadaisically and AK was subbed in and was a beast of defense. He guarded guys tight, swatted shots, and went to the basket hard. It was refreshing to see since those games are more and1 than nba. He's also been a favorite because he resembled Ivan Drago from Rocky IV so my old roommates and I would shout out the "I must break you" voice every time he blocked a shot.

Good luck Kirilenko! I hope you get whatever makes you happy. I'd definitely like to see you in a Warriors uniform.

Schwarzian Transform

2007-09-19T21:49:00.000-07:00

I like to remind myself of this Perl gem every so often. I always call it Schwarzian transform but it's actually Schwartzian transform.

#using schwarzian transform
#@sorted contains objects sorted by their distance
my @sorted=
map {$_->[1]}
sort { $a->[0] <=> $b->[0] }
map { [$_->distance(), $_ ] } @{$paObject};

map EXPRESSION, list returns a new list whose elements are the outcome of the expression
Example: @newArray = map {uc $_}, @array
is just
@array = qw(a b c d);
@newArray = ();
foreach my $temp (@array) {
   push @newArray, uc $temp;
} 

back to the example...
$paObject is a pointer to an array of objects.   The object has a method name distance() that returns an int
@{$paObject} dereferences the array.
working backwards,

map { [$_->distance(), $_ ] } @{$paObject}, creates an array like [[10,object reference 1],[2,object reference 2]]

and then feeds that into sort { $a->[0] <=> $b->[0] } which sorts according to the element at index 0

keeping with this example it yields [[2,object reference 2],[10, object reference 1]] 
which is fed into map {$_->[1]} which selects element at index 1

so now @sorted contains [object reference 2, object ref]

The benefit of the algorithm is that you only call the distance() method for each object once.  
If this method is computationally intensive and the list is large, the savings can be quite large.  
If your list is small, it's probably better to something like: @sorted =  {$a <=> $b} @unsorted

fixing IE checkbox padding/margin

2007-09-18T20:02:00.000-07:00

Support for CSS is built in to every modern browser so theoretically we shouldn't have to deal with too many layout issues. Unfortunately, a lot of browser specific quirks still exist. One I've dealt with recently is getting a checkbox to line up exactly to the left of a div. Setting the left margin and padding did not work in IE and I was going to try really drastic measures like setting float or absolute positioning until I found a really easy solution. To fix the extra padding for checkboxes in IE, we just need to give the checkbox a fixed width. I've summarized the results in the table with some screenshots. The markup I want to style is:


<div>
<p>some text before</p>
<input type="checkbox"/> checkbox 1<br/>
<input type="checkbox"/> checkbox 2<br/>
<p>some text after</p>
</div>

FireFox	IE	CSS	notes
		div {border: solid 1px #000;width:150px;} p {padding:0;margin:0;}	no styling on checkbox
		div {border: solid 1px #000;width:150px;} p {padding:0;margin:0;} input {padding-left:0;margin-left:0;}	style checkbox to have no left padding or margin. only Firefox behaves as expected
		div {border: solid 1px #000;width:150px;} p {padding:0;margin:0;} input {padding-left:0;margin-left:0;width:15px;}	added a fixed width and now IE7 looks fine too!

spring initbinder example that registers custom editor

2007-09-17T21:32:00.000-07:00

One of the nice features in Spring is the automatic databinding that occurs if you use one of the subclasses of BaseCommandController such as AbstractCommandController or SimpleFormController. By automatic databinding, I mean Spring initializing your command object based on the request parameters. The magic behind this feature is Spring's builtin property editors which do things like convert strings to integers. Sooner or later, however, you'll run into a situation where you need to do some custom databinding by registering your own PropertyEditor through the initBinder() method.

Suppose this is my command class:


class FootballPlayer {

static enum POSITION {RB, WR, QB, DB, LB}



POSITION _position;



public POSITION getPosition() {

return _position;

}



public void setPosition(POSITION position) {

_position = position;

}

}




So if my request parameter is ?positon=rb and I want my command class to have the correct enum set, do the following:
First, create a custom property editor.

class FootBallPlayerEditor extends PropertyEditorSupport {

public String getAsText() {

FootballPlayer editor = (FootballPlayer) getValue();

return editor.getPosition().name();

}



public void setAsText(final String text) {

setValue(FootballPlayer.POSITION.valueOf(text));

}

}



then override initBinder

protected void initBinder(HttpServletRequest request,

ServletRequestDataBinder binder) {

binder.registerCustomEditor(FootballPlayer.POSITION.class, new FootBallPlayerEditor());

}



And you're all set.

'Best Buy'er Beware

2007-09-16T23:06:00.000-07:00

I picked up an item at my local Best Buy and it rang up at 3 times the listed price. I politely mentioned to the cashier that this must be a price mistake so she led me to the aisle where she knew the item was being sold. The problem was that this wasn't the aisle where I picked up my product so I brought her to where I found the item. Sure enough, the price was the lower and so she sold me the item at the price I originally saw.

Seems odd that the same item was listed at different prices in two parts of the store. I found mine near the back and the other one was near the cashier. Either I got lucky or this is a Best Buy ploy to overcharge their customers. Moral of the story is twofold. One, do a walk through to see if your item is priced differently at a different spot in the store. Two, make sure you're watching when they ring up your order. Buyer beware.

And you thought Roger Goddell was a disciplanarian

2007-09-15T12:12:00.000-07:00

Roger Goddell is the commissioner of the NFL. Since he's taken over the top role about a year ago, he's suspended numerous players for 'conduct detrimental to the league' and now he's laid the law down among the coaches too.

The Patriots were caught taping the Jets as they were relaying defensive signals down to the field. The penalty for this action? A $500,000 fine for Belichick, head coach of the Patriots, a $250,000 fine for Bill Kraft, the owner of the Patriots, and the loss of a first round draft pick. The penalty wasn't too severe since Belichick wasn't suspended but it definitely wasn't a slap on the wrist either. I thought it was actually pretty fair since the lost of a first rounder could come back to bite them down the line and Belichick now has less cash to buy hooded sweatshirts and warmup pants.

Contrast this penalty with the hammer that was dropped when Mclaren was found in possession with Ferrari's playbook. A 100 million dollar fine and suspension from the team championships. That's right...100 miiiiiilliooooooon dollars!

Composite Comparator - Fun with generics and varargs

2007-09-14T19:10:00.000-07:00

A couple of new features in Java 5 are the additions of generics and varargs. We use generics a lot when returning a list from our DAOs but I've never had to implement a method that accepted varargs until I implemented this parent reviews page.

This is a typical page where a user may sort by date or by rating. Another feature is that a principal's review for a school always comes first. So the sorting is like so:

By Principal, then date descending
By Principal, then date ascending
By Principal, then rating, then descending

First thing I needed to do was create a couple of comparators. I used the static final comparator approach that that Josh Block talks about in Effective Java. I'll demonstrate with a baseball class. The nice thing about the composite comparator is that now I can create a comparator based on hr,rbi, and anything else I wanted to add later like sb, runs, etc.

Example:


public class BaseBallPlayer {

   private int _rbi;

   private int _hr;



   public BaseBallPlayer(int rbi, int hr) {

       _rbi = rbi;

       _hr = hr;

   }



   public static final Comparator HR_COMPARATOR = new Comparator<BaseBallPlayer>() {

       public int compare(BaseBallPlayer o1, BaseBallPlayer o2) {

           return o1.getRbi() - o2.getRbi();

       }

   };



   public static final Comparator RBI_COMPARATOR = new Comparator<BaseBallPlayer>() {

       public int compare(BaseBallPlayer o1, BaseBallPlayer o2) {

           return o1.getHr() - o2.getHr();

       }

   };



   public int getHr() {

       return _hr;

   }



   public int getRbi() {

       return _rbi;

   }

}

public class GenericComparator {

   public static <T>Comparator<T> createComparator(final Comparator<T>... c) {

       return new Comparator<T>() {

           public int compare(T r1, T r2) {

               int result = 0;

               for (Comparator<T> comp : c) {

                   result = comp.compare(r1, r2);

                   if (result != 0) {

                       break;

                   }

               }

               return result;

           }

       };

   }

}

   Comparator<BaseBallPlayer> HR_RBI_COMPARATOR = GenericComparator.<BaseBallPlayer>createComparator(

                   BaseBallPlayer.HR_COMPARATOR, BaseBallPlayer.RBI_COMPARATOR);

Gauntlet thrown

2007-09-13T16:13:00.000-07:00

I'm challenging myself to do a posting at least once a day for the next 30 days. This might not sound like a big deal to you super posters out there but have you seen my last posting date? It's been months and the updates have been sorely lacking.

I know what some of you may be thinking..."But Dave, I come here only for the thoroughly researched content and the essay like blog postings!"

Fear not loyal readers, quantity can equal quality. Plus, have you mistaken this blog for Paul Graham's? I'll continue to write is my stream of consciousness manner, but this time it will be a lot more often so expect a new blog entry for the next 30 days.

2007 yahoo fantasy baseball script

2007-04-12T19:37:00.000-07:00

update 4/24/2007
Script updated. Fixed issues 3 and 6 reported by Rodric Rabbah and lots of code refactoring. The script no longer automatically loads the window. You have to activate it by pressing on the blue "Show Freebie Stats" button on the top right.

update 4/18/2007
Script updated. Fixed issues 4 and 5 reported by Ethan Herbertson.

update 4/16/2007 To use the script, follow these 3 steps

download firefox
install greasemonkey extension
after installing greasemonkey, install the script

I just updated my fantasy baseball script for 2007. The most obvious change is that I incorporated the UI from my fantasy basketball script but there are also a bunch of under-the-hood fixes too. I go into some detail in those two blog postings on how the scripts are implemented so I won't recap any of that here but I will describe one bug fix. Stolen bases and homers are not part of the boxscore and they require extra parsing.

The stolen base string is of the form: D. Roberts 2 (2nd Inning off J. Bard), B. Bonds (3rd inning off J. Bard)

My old regular expression was a simple check to see if a players name was part of the string. The problem was that if I had J. Bard on my team, then he got a stolen base even though he was the one that allowed the steal. To fix the problem, I deleted everything in parentheses first.

That regular expression looks like: statLine = statLine.replace(/$[^$].+?\)/gi,''); The key is '?' which makes it a non greedy match and [^\)] which tells the engine when to stop.

Another thing that bothered me was calling mozilla's childNodes method on a table row returned some unexpected results. Instead of returning a list of td elements, it would also return some additional children. To remedy this annoyance, I call row.getElementsByTagName("TD") and then I get a list of td elements that I expect. That said, I'm much happier with Mozilla's handling of the dom api and the development plugins available in FireFox. My day job is as web developer for a high traffic site where we strive for pixel, perfect perfection between IE 6,7 and firefox so developing for just one browser was a nice break.

Install the script.

Best DNP description ever

2007-04-08T22:30:00.000-07:00

My good friend Johnny sent me a link to the best boxscore DNP description ever. We know Robert Horry is not the same player he used to be, but come on San Antonio, the man with six rings deserves a bit more respect.

inspiron 700m broken cd rom drive

2007-03-25T18:24:00.000-07:00

the cd rom drive on my laptop broke a couple of months ago and I never bothered to get it replaced since it never got any use. then yesterday I wanted to install turbotax. To workaround the missing drive, I mounted my desktop's cd rom drive and then accessed it through the network. problem solved!

2007 nba dunk contest

2007-02-20T10:14:00.000-08:00

Two thoughts entered my mind when I was watching the slam dunk competition:

Must be a direct correlation between the winner and the win pct of their team because the last couple of years players from the Celtics, Knicks, and Hawks have won.
Dwight Howard was robbed. That sticker dunk was original and difficult. He also had to do it in one attempt or else the shock factor would have been lost.

a fresh start

2007-02-16T10:49:00.000-08:00

It's a great time of year to be a baseball fan. The weather is warming up, your team has made some key offseason moves, and everyone is tied for first. I first became a fan in 1987 while rooting for my hometown team, the San Francisco Giants. I read every single baseball book in my elementary school library and became fascinated by larger than life personalities with colorful nicknames like the 'Sultan of Swat', the 'Say Hey Kid', and the 'Yankee Clipper'. I sneaked a walkman to bed and either fell asleep disappointed or happy depending on the outcome of that night's game. It remains the only sport where the radio broadcast is just as enjoyable as the television broadcast.

I also like the team and individual aspects of the game. A pitcher relies on the players playing behind him to make plays and to score runs. Yet, at its core is the battle between the hitter making solid contact against a pitcher. There's also the little things about the game that just work. Ninety feet is the perfect distance between first and home since the speediest runner can still be thrown out when defense is played perfectly. That distance is as cool to me as closures. While the infield is of uniform dimension, the outfield can vary widely between stadium to stadium. A ball crushed to right-center will be an out in Pac Bell Park but a homer in practically every other stadium. While park designers may embellish these idiosyncacies, *cough Minute Maid Park hill*, there's no denying the beauty of the Old Tiger Stadium and Fenway Park.

The optimist in me believes the Giants can make a playoff push bolstered by their improved pitching. Zito will assume ace responsibilties and that new windup may give him that extra zip and movement. A seasoned Matt Cain, aka Bobby Hill, will have learned how to pitch, Noah Lowry might be able to fool hitters once more, and Matt Morris will be able to avoid that one big inning blowup he always has. The fifth starter, whether it is a renewed Russ Ortiz or rookie sensations Linceum or Sanchez will surprise the league by storm and Benitez will be in shape and less of a headcase.

On the offensive end, the old guys will be injury free. Barry Bonds will break Hank's record by early August so the team can concentrate on the playoff push and Dave Roberts will be that leadoff hitter we haven't had since Brett Butler. Rich Aurilia pounds 40 homers and shows Seattle fans why they once signed him to big bucks.

Its a great time to be a baseball fan. We're currently in first place.

yahoo sports redesign

2007-02-02T23:56:00.000-08:00

Wow, yahoo sports has had a major makeover. About a year ago, I blogged about espn's redesign and how it was an upgrade. I feel similar about the new yahoo site but not as excited since espn was the real trailblazer with their css based layout.

Both sites look really similar now with a huge pic on the LHS and links to articles on the RHS. But there are some improvements that differentiate the designs. Yahoo's links to articles has multiple tabs "Headlines", "Rumors", and "Most Popular" which allows for a bunch of links above the fold. Yahoo also lists the top fantasy performers on the front page which is a nice touch for fantasy fanatics like myself. That said, the yahoo designers had a year to come up with new stuff and the fact that the site kinda looks like espn really speaks volumes about the espn design.

The only site I've been visiting for years that remains the same is craigslist.

my first ebay purchase, the kinesis maxim ergonomic keyboard

2007-01-12T15:21:00.000-08:00

"Welcome to the late nineties!", my coworker Chris jokingly exclaimed when I mentioned during lunch that I had made my first ebay purchase.

I've been checking out keyboards without number pads and finally splurged on the maxim kinesis. Other models I considered were keyovation's goldtouch, happy hacking, and some lower-priced ones: logitech model made for the playstation 2, and zippy. I picked the maxim because I found a new one available for a great price on ebay. I wanted a keyboard without a number pad so that I wouldn't have to reach as far for the mouse.

The keys on the maxim are a bit stickier than my old dell quietkey, but I soon got used to the feeling. The maxim has some neat variable split and tilt features that's supposed to reduce hand fatigue. The purchase of the maxim also gave me a great excuse to try the Dvorak keyboard layout. Since I was getting accustomed to the keyboard anyway, I decided to up the ante. As an engineer, I've always been impressed with the efficiency of the layout and plus it's another way to geek out.

Dvorak studied letter frequency and designed the layout to maximize hand alternation and minimize movement away from the home keys. The qwerty layout, on the other hand, was designed to minimize the jamming of keys back when folks still used typewriters.

Learning Dvorak takes patience and practice. The first couple of days were a mess. It took me forever to write a simple email. I finally had to switch back to qwerty to get any work done. But I've never been one to give up, so I practiced at home and by day four I was able to make the switch at work.

I've been using the new layout for two weeks now and I'm finally competent. My WPM is around 30 and I expect that to improve since there's usually a month long acclimation process. One benefit I'm already enjoying is how fresh my hands and wrists feel at the end of the day. This was my main motivation for making the switch and so far it's paying off.

snowboarding at Donner Ski Ranch

2007-01-12T14:23:00.000-08:00

I finally hit the slopes last week and it was a great time out. We went up to Donner Ski Ranch which is a perfect mountain for beginner and intermediate riders. Perfect because the lift lines are short, the price is cheap, and the slopes aren't too steep. I concentrated on perfecting my turns and by the end of the day I was leaving some nice 'S' tracks. I had a couple of hard falls but that was offset by some good runs.

I also got to use my new Leedom helmet with built-in speakers for my mp3 player. The music kept my energy level up while I was practicing toe-side tracks. The helmet had lots of vents and was really lightweight. I bought it for $75 from costco online and am really happy with it.

I definitely see and feel the improvement and look forward to making a few more trips this season. Here are more pics courtesy of Simon.

wisdom teeth and remembering passwords

2006-12-14T13:29:00.000-08:00

This Monday, I had all four of my wisdom teeth removed. I went the local anesthesia route so I was awake during the procedure while my oral surgeon drilled and pulled at my molars. The main reason I didn't want to be put under was so that I could eat a ton of food before the removal. Even though listening to your teeth crack isn't the most pleasant experience, having a full stomach and not being nauseous afterward is a good trade off. I still can't really chew on food so I've been eating jook, mashed potatoes, and soup for the past few days. Most of the bleeding and pain is gone by day 2 but my cheeks are still pretty swollen. I look like a chipmunk blowing a bubble minus the bubble. I've had a couple of days to rest and so I've been reading, watching tv, surfing the web, and now blogging.

In fact, this entry was triggered by the security breach at reddit.com. A hard drive was stolen and reddit recommends that passwords be changed. Because of the recommendation, commenters assume that the passwords were stored as plain text. Another top story on reddit, was an article on weak passwords written by security guru Bruce Schneier. So what better time that to share how I deal with passwords. For freebie or temp accounts that have no personal info, I use a simple password. For accounts that need better security, I remember one main password and then use that main password and a unique key that is based on the web url and some other string to generate a password. I use this standalone, javascript password generator which I have saved on my local machine.

why I like perl

2006-11-11T12:33:00.000-08:00

At work, we're primarily a java shop but yesterday I had to edit a perl module that is used by our editorial department. I looked at an existing function and wanted to refactor it since there was a lot of code duplication. To do so, I needed to dynamically generate an sql statement. The statement was of the form:

Insert into table (col1, col2, .. colx) values (?,?,...,?)
or Replace into table (col1, col2, .. colx) values (?,?,...,?)

Array @columns contains the name of the columns. so to generate the string "col1, col2, col3", it's just: $string1 = join ',', @columns

now to generate the "?,?,?" strings, it's just $string2 = join ',', map '?', (1.. scalar @columns)

yahoo fantasy basketball live scoring greasemonkey script

2006-10-31T21:41:00.000-08:00

Updated 2007-Nov-04: Updated the script to calculate team totals and also maintain roster order as it appears on management page.

Updated 2006-Nov-05: I cleaned up the boxscore processing code to implement a feature my friend Simon requested: Viewing the live stats of other managers' teams. Be sure to update the script to use the new features.

I didn't write a baseball script until after the all-star break and I passed on writing a football script since it wouldn't save me much time. But tonight I finished a greasemonkey script that provides live scoring for yahoo's fantasy basketball leagues.

To use it, download the newest version of greasemonkey and then install the script. Yahoo provides live scoring until nov 11 so you've got plenty of time to dissect the code to make sure it's safe :) Don't worry, unless someone has hacked into my code repo (very unlikely since it's hosted by google), the script does not contain nefarious code. That's always my biggest concern when installing scripts written by others since it's so easy to place code then sends data to other remote servers. That said, the script is provided as is so if it drops Kobe or Lebron from your team then oops :) I'm only kidding...I've tested it on windows xp prof running firefox 2 and the latest version of greasemonkey and have had no problems.

Here's a screenshot of the script in action. Note that the bench players have a darker background color:

The logic of the script was similar to the baseball one. First, I parsed out the boxscore links and then I made an ajax request to retrieve the contents of each boxscore. Finally, I used xpath to find the stats for my player. The techniques I used for xpath and ajax are explained in detail here. Although the business logic for the script is similar, I employed a vastly different view for presentation. Instead of embedding the stats directly into the existing table cells like in the baseball script, I placed the basketball stats inside a modal window.

Using a modal window greatly simplified my code because I didn't have to keep track of each stat and its corresponding display cell. Instead, I could copy the row node from the boxscore and place it directly into my modal window without any extra processing. I also noticed that my fantasy football team's page on sunday night no longer had placeholders for stats. The modal will bypass this problem should yahoo implement something similar for fantasy basketball.

There are a couple of different methods to implement modals. The contents of a modal can be from an iframe or a hidden div. My script uses the hidden div technique since the stats are added to the DOM on the fly. If you're interested in implementing something similar, a must read is this blog posting on modal windows. I used that technique along with some others that I picked up while implementing modals at work. In fact, the autocentering code is from an enhanced, iframe-based modal library called submodal that my boss Todd has greatly enhanced.

One bug that took me a while to track down was that some players made multiple ajax requests. I added logging using greasemonkey's GM_log api and discovered it was related to my xpath expression:
//table[@id='statTable0']/tbody/tr/td[@class='player']/div/a[@href]

this expression finds all player profile links but it also found links to the 'breaking news' about the player. I fixed the problem by adding an extra check for the a tag: a[@class='name' and @href] ... problem solved :)

Enjoy the script!

python stats functions for netflix prize

2006-10-30T11:41:00.000-08:00

A coworker and I entered the netflix prize which is a competition to improve netflix's recommendation engine. The grand prize is 1 million dollars if your algorithm can beat netflix's by 10 percent. The task is to predict how a user will rate a movie based on previous ratings. The past results dataset is around 2 gigs uncompressed and it works out to 100480507 ratings for 17700 movies. I won't disclose our algorithms just yet but here's something I will share. It's a couple of python statistical functions that I whipped up that calculate the mean, std deviation, kurtosis, and skewness of a list. One requirement is that you import math because I use the sqrt and pow functions in that package.

There's already a python stats module that you should use if you plan to do some heavy lifting with stats in python but if you're just looking for something a bit simpler then here are the functions. They make use of a neat python feature called list comprehension that makes the code a lot easier to write and understand. There's no perl counterpart to this built-in language feature, but the same can be accomplished with a map and an anonymous function. One of my main motivations for working on this project is learning python.

def mean(x):
        """Find mean of a list"""
        return sum(x)/len(x)


def stddev(x,mean):
        """Find standard deviation of a list"""
        return sqrt(sum([pow((val-mean),2) for val in x])/len(x))

def skewness(x,mean,stddev):
        """Find skewness of a list"""
        return sum([pow(val-mean,3) for val in x])/(len(x)*pow(stddev,3))


def kurtosis(x,mean,stddev):
        """find kurtosis of a list"""
        return (sum([pow(val-mean,4) for val in x]) / ((len(x)-1) * pow(stddev,4))) - 3

After I calculated these stats for the movies, I loaded the information into a database and played around with the data.

The fifteen most rated movies are:
mysql> select name,ratings from movie order by ratings desc limit 15;
+--------------------------------------------------------+---------+
| name                                                   | ratings |
+--------------------------------------------------------+---------+
| Miss Congeniality                                      |  232944 |
| Independence Day                                       |  216596 |
| The Patriot                                            |  200832 |
| The Day After Tomorrow                                 |  196397 |
| Pirates of the Caribbean: The Curse of the Black Pearl |  193941 |
| Pretty Woman                                           |  193295 |
| Forrest Gump                                           |  181508 |
| The Green Mile                                         |  181426 |
| Con Air                                                |  178068 |
| Twister                                                |  177556 |
| Sweet Home Alabama                                     |  176539 |
| Pearl Harbor                                           |  173596 |
| Armageddon                                             |  171991 |
| The Rock                                               |  164792 |
| What Women Want                                        |  162597 |
+--------------------------------------------------------+---------+

the movies with the highest mean are:
mysql> select name,mean from movie order by mean desc limit 15;
+---------------------------------------------------------------------+---------------+
| name                                                                | mean          |
+---------------------------------------------------------------------+---------------+
| Lord of the Rings: The Return of the King: Extended Edition         | 4.72326993942 |
| The Lord of the Rings: The Fellowship of the Ring: Extended Edition | 4.71661090851 |
| Lord of the Rings: The Two Towers: Extended Edition                 | 4.70261096954 |
| Lost: Season 1                                                      | 4.67098903656 |
| Battlestar Galactica: Season 1                                      | 4.63880920410 |
| Fullmetal Alchemist                                                 | 4.60502147675 |
| Trailer Park Boys: Season 4                                         | 4.59999990463 |
| Trailer Park Boys: Season 3                                         | 4.59999990463 |
| Tenchi Muyo! Ryo Ohki                                               | 4.59550571442 |
| The Shawshank Redemption: Special Edition                           | 4.59338378906 |
| Veronica Mars: Season 1                                             | 4.59208393097 |
| Ghost in the Shell: Stand Alone Complex: 2nd Gig                    | 4.58636379242 |
| Arrested Development: Season 2                                      | 4.58238935471 |
| The Simpsons: Season 6                                              | 4.58129596710 |
| Inu-Yasha                                                           | 4.55443429947 |
+---------------------------------------------------------------------+---------------+

the movies with the lowest mean:
mysql> select name,mean from movie order by mean asc limit 15;
+----------------------------------+---------------+
| name                             | mean          |
+----------------------------------+---------------+
| Avia Vampire Hunter              | 1.28787875175 |
| Zodiac Killer                    | 1.34602081776 |
| Alone in a Haunted House         | 1.37560975552 |
| Vampire Assassins                | 1.39676117897 |
| Absolution                       | 1.39999997616 |
| The Worst Horror Movie Ever Made | 1.39999997616 |
| Ax 'Em                           | 1.42222225666 |
| Dark Harvest 2: The Maize        | 1.45238089561 |
| Half-Caste                       | 1.48739492893 |
| The Horror Within                | 1.49624061584 |
| Vampires vs. Zombies             | 1.49659860134 |
| The Bogus Witch Project          | 1.49775779247 |
| Rise of the Undead               | 1.50295853615 |
| Vampiyaz                         | 1.50391650200 |
| Underground Comedy Movie         | 1.50503361225 |
+----------------------------------+---------------+

high standard deviation
mysql> select name,std_dev from movie where ratings > 500 order by std_dev desc limit 15;
+----------------------------------------+---------------+
| name                                   | std_dev       |
+----------------------------------------+---------------+
| 'N Sync: Live at Madison Square Garden | 1.61672019958 |
| Dragon Ball Z: Vol. 17: Super Saiyan   | 1.58626019955 |
| Dragon Ball Z: Trunks Saga             | 1.58625769615 |
| Sailor Moon R: The Promise of the Rose | 1.54338657856 |
| Princess Nine                          | 1.54320299625 |
| Big Brother 3                          | 1.52889740467 |
| Family Guy: Live in Las Vegas          | 1.51085150242 |
| Dragon Ball Z: Imperfect Cell Saga     | 1.51082730293 |
| Cher: Live in Concert                  | 1.50970792770 |
| Cardcaptor Sakura                      | 1.50106477737 |
| Tupac Shakur: Before I Wake            | 1.49814224243 |
| Slayers Try DVD Collection             | 1.48955285549 |
| Dragon Ball Z: The World's Strongest   | 1.48337769508 |
| Dragon Ball: The Saga of Goku          | 1.48244822025 |
| Dragon Ball Z: Super Android 13        | 1.47919940948 |
+----------------------------------------+---------------+

high kurtosis (=deviation due to extreme values)
mysql> select name,kurtosis from movie where ratings > 500 order by kurtosis desc limit 15;
+---------------------------------------------------------------------+---------------+
| name                                                                | kurtosis      |
+---------------------------------------------------------------------+---------------+
| Lord of the Rings: The Return of the King: Extended Edition         | 8.98888015747 |
| The Lord of the Rings: The Fellowship of the Ring: Extended Edition | 8.81478309631 |
| Lost: Season 1                                                      | 8.58331871033 |
| Lord of the Rings: The Two Towers: Extended Edition                 | 7.96247339249 |
| Veronica Mars: Season 1                                             | 7.01007556915 |
| Battlestar Galactica: Season 1                                      | 6.57994556427 |
| Arrested Development: Season 2                                      | 6.34450483322 |
| Fullmetal Alchemist                                                 | 6.30700731277 |
| The Simpsons: Season 6                                              | 5.94859170914 |
| Inu-Yasha                                                           | 5.37345218658 |
| The Simpsons: Season 5                                              | 5.32154130936 |
| Fruits Basket                                                       | 5.32094144821 |
| House                                                               | 5.26964759827 |
| Family Guy: Vol. 2: Season 3                                        | 5.23487997055 |
| The Simpsons: Season 4                                              | 5.13775539398 |
+---------------------------------------------------------------------+---------------+

select name,skewness from movie where ratings > 500 order by skewness desc limit 15;
high skewness (long positive tail=random people who voted a bad movie good)
+--------------------------------------------------------------------------+---------------+
| name                                                                     | skewness      |
+--------------------------------------------------------------------------+---------------+
| Underground Comedy Movie                                                 | 2.18097925186 |
| Ben & Arthur                                                             | 2.11108279228 |
| Visions of Sugarplums                                                    | 1.73864912987 |
| Dracula 3000                                                             | 1.56867814064 |
| Guilty by Association                                                    | 1.49578213692 |
| Survivor Exposed                                                         | 1.48025572300 |
| Leonard Part 6                                                           | 1.40650367737 |
| Homo Heights                                                             | 1.36938285828 |
| Going Overboard                                                          | 1.35962140560 |
| Shanghai Surprise                                                        | 1.27664005756 |
| Sopranos Unauthorized: Shooting Sites Uncovered                          | 1.27566981316 |
| Twentynine Palms                                                         | 1.20583796501 |
| The Life                                                                 | 1.15580248833 |
| National Lampoon's Christmas Vacation 2: Cousin Eddie's Island Adventure | 1.05684077740 |
| Glitter                                                                  | 1.04486155510 |
+--------------------------------------------------------------------------+---------------+

weekend in big sur

2006-09-27T17:28:00.000-07:00

My friends and I spent the weekend at big sur. We camped one night at Big Sur Campground. Unlike a lot of camgrounds, this one was in the midst of towering redwoods and it was truly a spectacular experience waking up and seeing the sunlight filtering through branches and coastal fog. The campground also had hot showers and even a playground for kids. Simon's coworker who made the reservation said that the premium sites by the river were booked 5 years in advance so plan ahead if you want to stay at the same place :)

Big Sur is along California's central coast just south of Carmel and Monterey. Mid september is our summertime and the weather didn't disappoint. It was high 70s, low humidity, and clear skies all around. We drove down saturday morning, pitched our tent and then went on a couple of hikes in Julia Pfeiffer state park.

the first one was Pfeiffer Falls which is around a mile roundtrip and ends at a nice 60 foot waterfall:

The second trail we went on is Valley View which had a nice view of the valley. That one was about a mile or so:

The last one we did was Buzzard's roost which is around 5 miles round trip. It starts with a nice walk along the big sur river

and then climbs up to around 800 ft at the summit. The path to the summit was quite overgrown so we had to squeeze past some chapparal. There was actually an old shed up there and also a some rock outcroppings to sit on. Here's the view west toward the the pacific ocean:

View east toward the Santa Lucia mountains:

it's a boy!

2006-09-26T10:47:00.000-07:00

Lots have happened since my last update. The biggest annoucement is that my sister gave birth to the cutest little baby boy, Josh Michael on September 7th. I've been visiting the little guy quite often and he looks more adorable every time I see him. He's my first nephew and I look forward to going to Giants and Niners games with him. He's also displaying tremendous strength in his left arm so maybe he'll be the Sandy Koufax or Steve Carlton of his generation. That shouldn't be too much pressure to place on the little tyke :) Notice I'm already picking a 'safe' sport like baseball for him instead of bone crunching football.

The niners are doing more crunching this year, even though they're 1-2. I doubt they will make the playoffs but at least they are somewhat competitive which is saying something based on what I saw last year. Getting rid of Barlow was a good move and so was getting Norv Turner. His track record with young quarterbacks is pretty good and I'm already seeing more offensive plays that utilize Alex Smith's mobility. Last year, Alex Smith was primarily a pocket passer and this year I'm seeing rollouts and designed run plays. Our O-line is providing more protection too..we finally got rid of Newberry and even though I'm sad to see him go it was time. Sad because he was the last connection to Bob Mckitttrick but glad because he couldn't possibly hold up on two shot knees with no cartilege. My fantasy team is currently 0-3 but to be fair I gotta say that I lost 2 of them by a combine 4 points and the most recent defeat came against a Cincinnati defense that scored over 25 points. WTF?? I hope my luck changes. I got Portis and Cooley in my starting lineup so hopefully Washington's offensive explosion continues.

I started the football fantasy greasemonkey script but just haven't found the motivation to finish it. Unlike my baseball version which saved me lots of time checking out box scores everyday, football games don't occur everyday. I already pour over the boxscores anyway so I'm not really saving that time finding out how my players did. I have gotten some good feedback from the baseball one so I'll leave it to those users...anyone want to see a football one? or should we just wait 'til basketball season starts? I'm definitely doing a basketball one.

greasemonkey fantasy baseball script

2006-08-13T22:48:00.000-07:00

update: 4/12/2007 Updated script for 2007 season. update: 8/21/2006 added HR and SB support and also optimized some xpath processing.

I'm a big sports fan and one of my favorite hobbies is playing fantasy sports games on yahoo. For the season's first couple of weeks, yahoo provides real time stats for all your players. But after that, they package this feature into their 'premium services'. The only way to get live scoring is to pay for it or to click on each box score link and then scan for your player. If the game has already ended, it's actually two clicks since the initial link takes you to the game recap.

This is can be quite time consuming if you manage multiple teams but you really have no choice. ...that is, no choice until now: This weekend, I wrote a greasemonkey script that iterates through all the boxscores and parses out your player's stats for the night. Once the script has finished running, you'll have live scoring!

I've written it for default baseball leagues but a football one should follow shortly once I get around to drafting my football team. here's a screenshot of how my players performed tonight:

You can download the script here: http://freebiestats.googlecode.com/svn/trunk/baseball.user.js

Feel free to send feature requests or bug reports here: http://code.google.com/p/freebiestats/issues/list

This is my first greasemonkey script and I'm quite impressed with the whole framework. After the page renders, greasemonkey kicks in and can manipulate any part of the dom. The flow of my script is as follows:

get the box score link for each player
visit that link and get the player's stats
update page with the stats

Firefox has a kick ass xpath implementation so finding all the box score links was pretty easy. It helps that my xpath was pretty sharp from all the xml feed projects I've been doing at work. The xpath for the batters is:
//table[@id='statTable0']/tbody/tr/td[@class='player']/div/a[@href]

The only time consuming part when developing this script was figuring a way to visit and process the box score pages. I definitely wanted to create a dom for the box score page so that I could use the aforementioned kick ass xpath functionality but my first attempt using greasemonkey's GM_xmlhttpRequest function wasn't too promising. I had no luck transforming the raw html returned by GM_xmlhttpRequest to a dom using DomParser. This must be a common problem because a google search yielded this workaround in greaseblog. That solution:

var parser = new XPCNativeWrapper(window, "DOMParser").DOMParser();
alert(parser.parseFromString("", "text/xml"));
seemed a bit hackish and it would only work for xml documents. Since the box score page is not xhtml valid or even html valid for that matter, this was not a viable option.

My second attempt was to use an iframe whose src was the box score link. This seemed to work but then I didn't have permission to access the dom directly. I kept getting permission problems. I'm guessing that firefox disabled this feature to combat cross site scripting issues. If you really want to use an iframe, you'll have to use this solution from youngpup.

My solution was to go back to using GM_xmlhttpRequest to get the raw html of the box score, saving contents within the body tag into a string, and then creating a hidden div on the current page whose innerHTML was set to that string. The box score page was now part of the current page's dom and I was able to write xpath expressions.

after that problem was solved, everything else came together rather quickly. The script is definitely an EA, 0.1 version since it's tightly coupled to how yahoo publishes box scores and fantasy manager pages and HR and SB is not one of the available stats. Ultimately, I plan to resuse a lot of this code in other fantasy games and automatically determing the order of the stats on the management page and the box score page will be a common problem that will need to be solved.

I'm currently hosting this project using google code which has an issue tracker and uses subversion as its version control system. My javascript editor is intellij idea.

ResourceBundle bungle

2006-07-12T16:16:00.000-07:00

Updated 7/13: used google pages as my online wysiwyg to generate valid html for xml files

We use SpringMVC and use the built-in validator support. I had a problem parsing in a parameter for an error message.

my applicationcontext contained:

the line in my resource bundle,app.property, was:
error_key=What's up, {0}?
the validator call was:
errors.reject(errorcode, new Object[] {"doc"}, defaultMessage);

My jspx file looked like this:

      <spring:hasbinderrors name="mycommand">
          <div class="error">
              <c:foreach var="error" items="${errors.allErrors}">
                  <li><spring:message arguments="${error.arguments}" code="${error.code}"></spring:message></li>
              </c:foreach>
          </div></spring:hasbinderrors>

I expected this to show up in my final output: What's up, doc? Instead, the parameter was never parsed in. It only displayed: What's up, {0}?

Like always, the answer was found in the javadocs. The parameter binding is backed by java.text.MessageFormat and what happens is that a single quote, ' , stops any binding. So to solve the problem, use two quotes '' to represent a single quote.

fmt:formatNumber rounding behavior

2006-07-06T00:04:00.000-07:00

I was helping a coworker with a rounding "bug" earlier today. On one of our jspx pages, we need to round some numbers up. But she ran into a strange problem because 200.5 rounded to 200 but 21.5 rounded to 22. She thought the problem was hibernate related but a couple of quick unit tests revealed the real culprit: jstl's fmt:formatNumber tag. It's actually not a bug at all because according to the javadocs, it relies on the following for rounding:

ROUND_HALF_EVEN

public static final int ROUND_HALF_EVEN

Rounding mode to round towards the "nearest neighbor" unless both neighbors are equidistant, in which case, round towards the even neighbor. Behaves as for ROUND_HALF_UP if the digit to the left of the discarded fraction is odd; behaves as for ROUND_HALF_DOWN if it's even. Note that this is the rounding mode that minimizes cumulative error when applied repeatedly over a sequence of calculations.

Unfortunately, there's no way to change the behavior to ROUND_HALF_UP which is what we wanted. I'll send a note to the jstl mailing list to see if anyone else would like this feature and if so submit a patch.

*Update Before I sent that email to the mailing list, I did more digging and discovered that fmt:formatNumber is backed by DecimalFormat. As of Java 5, DecimalFormat does not support specifying a rounding mode other than the default ROUND_HALF_EVEN. The good news is that setRoundingMode was introduced in Java 6 to solve this exact problem. The bad news is that it'll be a while before java 6 is out of beta and this change gets propogated down to fmt:formatNumber. So in the meantime, the best thing to do is write your own tag is you require a rounding mode other than ROUND_HALF_EVEN.

back in business

2006-03-30T23:39:00.000-08:00

Ucla is going back to the final four. Our storied basketball program is back in business and returns to its familiar place among the hoops elite. Like enjoying a good wine, I needed to savor the memories of defense and clutch plays before penning my ode to this tournament run.

When the brackets were annouced and Ucla claimed a number 2 seed I knew we would make it to the final four. Getting past Gonzaga was a real test of my faith but these kids came through. We unleashed a suffocating defense and Coach Howland managed the game to perfection.

Thanks to streaming video from march madness on demand, I replayed the clutch Hollins free throws and the ensuing steal to find some great coaching nuggets. After the free throws, Hollins was quickly subbed out for a quicker Collison. Collison's responsibility was to cover deep which freed Mbah a Moute to cover the player inbounding the ball and allowed Bozeman to switch over to Batista. The ball was inbounded to Morrison who made a great pass to Batista. Batista was quickly swarmed by Farmar and Bozeman was able to knock the ball loose. Howland was in the game the entire time..perhaps it was coincidence that the play turned out the way it did, or maybe it's just because the guy can flat out coach.

It's so great to see the maturation of the players and the team's level of playthis past season. Ryan Hollins just became a millionaire by his play in this tournament alone. It's great to watch and I'm looking forward to this weekend's game.

Idea 5.2 and subversion installed

2006-03-01T23:08:00.000-08:00

I've finally gotten around to starting my XmlEditor project. I plan to create a swing app that validates an xml file against an xsd. I also plan to allow the app to allow editing of the file to fix errors. This project will introduce me to swing and also to the java nio classes. I'm not sure if it's necessary to use nio but it's something I've been meaning to learn/use for a loooong time so we'll see how it goes.

Before I start my project, I want to make sure I have a revision control system (as encouraged in the pragmatic programming series) on my machine so I downloaded and installed subversion. The new version of Idea has subversion support built in so I upgraded from 4.5 to 5.1. It was a breeze to import settings and existing projects. I chose subversion because I already know cvs and wanted to try something new. I've heard the branching system makes a lot more sense and is a lot faster but I doubt I'll need branches for this small project. It took me around 20 days to finally start this project so let's see when it's finally finished :)

snowboarding at sugarbowl

2006-02-12T18:26:00.000-08:00

It was first my boarding session of the season (my 3rd time ever) yesterday and I really enjoyed it. We rented a minivan and made it to berkeley around 6 to pick up Jonathan. We pitted at a McDonald's in davis and arrived at sugarbowl at around 9:30pm.

I'm happy to say that I've vastly improved from my previous two snowboarding trips. I can now go down a steep hill on my board rather than my butt. My s-curves are coming around and I hope to go a couple more times this season to practice them. I'm defeinitely still a novice but getting better every time I get some runs in. I brought along my mp3 player and that along with a couple of cliff bars kept my energy level up until closing time at 4ish. I fell hard several times so today I woke up pretty sore but it was still a good trip.

planned programming projects

2006-02-10T16:16:00.000-08:00

I consider myself lucky to deal with interesting programming challenges at work. It definitely keeps me on my toes to know that web applications I develop will be used by millions of our customers. Just from work programming, I've become quite adept with Perl, MySQL, and Java.

But to keep pace with all the interesting technologies and advances in this industry, it's important to hack on items outside of work too. Reading blogs definitely keeps you updated, but there's only so much you can do by reading...sometimes doing is the best teacher.

I've got two projects that I plan to work on in the coming weeks. The first is to develop a Ruby on Rails webapp that will be used to access this vast trove of baseball statistics. The second project is a Swing app that will be used to validate XML files against a user specified XSD. I deal with large XML files all the time and I've always wanted such a tool. I'm leaning toward the Swing project first but I might try working on both at the same time. Stay tuned for the code...it'll either be beerware or will be simple like freebsd's license.

IE printing problems and how to fix them

2006-02-09T16:01:00.000-08:00

I work at a pertty high traffic site and we get immediate feedback from users if a problem occurs. A popular section of the site is our library of articles. Like other sites, we offer a "print view" of the article that does not contain ads, headers, and footers. Our IE uesrs, however, were having problems printing the article out on their printers. The rightmost side of the content was being chopped off even though it looked fine on the screen. I wasn't aware of this problem since I'm a firefox user but it made its way to my JIRA task list under blocker status so I promptly investigated the problem.

A quick search on Google revealed that this was a common IE printing problem and after refining the search I discovered the IE printing problem described by the IE programmers themselves. The problem was partly due to HTML designers but IE also deserves some of the blame as well since printing is fine on Firefox. I delved into our html markup and gutted a lot of unnecessary tables and invisible images used to control width that really seems prehistoric in these web 2.0 times. I pushed the fix live quickly and maybe..just maybe... saved some paper from having to be recycled or tossed out.

if you ever run into a problem printing a page out in IE, and you don't want to download firefox (why not?), these are the links to third party solutions that the IE bloggers recommended:

pair programming, sponsored links, and fuzzy matching

2006-02-07T22:23:00.000-08:00

I got my ass kicked by the flu last week. Last tuesday morning I felt more tired than normal rolling out of bed and it got progressively worse throughout the day. I left work about an hour early but thanks to the prompt and speedy service of Muni, I arrived home at the same time. I waited for about 45 minutes in the cold for a bus to come and barely managed to squeeze on. I wish it were hyperbole but I literally was standing in the stairwell of the bus next to a cell phone yapping fool. When I got back home I downed some Tylenol and from then til about Friday morning I was in a haze. I promptly caught a cold after the flu and it wasn't until yesterday that I was feeling my normal self.

My first project coming back was helping my coworker pull some data to give to a client and that was actually a nice one since it was relatively easy and got me back in the flow of code. We've been trying to do more pair programming and I'm all for it since more eyes means less errors. Retrieving multiple datasets that aren't stored consistently isn't exactly the most exciting project either so having someone to discuss different strategies and keep the focus on usually speeds things up. I've pair programmed before and I found it to be very fast when both programmers are at similar skills levels. The converse is that it takes twice as long when you're paired with a rookie but that's also a great way for some knowledge share provided the rookie isn't overwhelmed ala Michael Jordan and Kwame Brown.

That took up most of my morning and the rest of the day I programmed solo and implemented a sponsored link section on our site for the ad team. After I finished that, I worked on fuzzy macthing private schools that our staff had manually entered into our database against a set of data provided by the government. Initially I was going to use mysql's built in full text queries on the name and the addresses but the queries required can get a bit tricky and I wasn't ready to implement them in hibernate's query language.

Instead my algorithm was to go through each school in the government's file and then grab all schools in our database that's located in the same city. I computed the Levenhstein distances on the school name and also the street address and that gave me some really good matches for distances of 10-20.

espn's redesign + my first homepage

2006-01-18T22:51:00.000-08:00

Espn's front page has gone through another design iteration. This is how it used to look. It was only a matter of time since they've been testing a similar look on their sport specific homepages. I like it a lot..check it out when you get a chance...the days of a left navigation bar are slowly fading away. Since it's been eliminated, more space is available for content. The readability of the top story and headlines, the main content I go to the site for, are much easier to read because of the extra space and size. On the tech side, I know they were early adopters of css and xhtml so I'm guessing that this design change wasn't as difficult as the first time they went xhtml and css.

My approach to css is quite pragmatic. I use it whenever possible but If I have to employ too many hacks to get it to look right in a certain browser, I will ditch it for a universal solution like using tables for layout. In fact, when I coded up the test score and census display for work, I went with tables because I needed total confidence that the achievement pages would render correctly for older browsers and drawing complex ,vertical bar charts whose display is CSS based is impossible without hacks.

on a separate note, I found my old college webpage while looking at the early espn pages. be sure to check out the random quote that appears on the bottom of the page.

Politcal spin in today's classrooms more common than the freshman 15?

2006-01-18T12:36:00.000-08:00

I came across the most absurb topic today while perusing sfgate.com: "Radical" UCLA professors targeted by alumni group" The gist of the article is that some alumni group is encouraging current students to document politically biased salvos during lecture time in the form of a monetary finder's fee. I'm not really against students who want to make an extra buck in this manner since I too was once a poor, ramen eating UCLA bruin, hence robobruin, I'm more against corporations/individuals who deem it necessary to patrol a professor's political views.

This is college. Students are exposed to much greater ills than a professor spending a bit too much time on Bush and the war in Iraq. Heck, Jon Stewart probably has a greater influence and a more captivating presence on a college student's political views than his professor. We aren't talking about a bunch of impressionable kids like in Dead Poet's Society. Let's give UCLA students some credit here...these are some of the smartest young adults in the country and they definitely can differentiate textbook analysis and political slant during a lecture.

Let's say the argument is not whether a professor is inculcating a student with his political views but rather that he's just wasting time talking about an irrelevant subject. I checked out some of the targeted professors and they mostly teach political/social science/law classes. Are you telling me that current political affairs are off topic? Isn't there great debate about the legality of guantanamo bay or wiretapping US citizens by way of the Patriot Act? If the professor is just there to repeat what's in a textbook why go to class at all? His job is to expand and add to the textbook and that may involve some of his political views. I'm ok with that...I've never been brainwashed by a professor..they're some of the smartest guys/gals in the world but definitely not omniscient.

I'm just disappointed no Boelter hall professors were mentioned. I wouldn't have minded hearing some political spin during my time as an undergrad in the engineering building. That might have made those required fluid dynamic classes a bit more palpable. And it may have been fun to see the words 'Bush' and 'Liberal' used to demonstrate the Levenshtein distance formula. A computer science prof could even have slipped John Kerry in a state diagram that led to a flip flopping of results.

24

2006-01-17T00:41:00.000-08:00

with shows like "lost" and "24", I almost can understand why people don't subscribe to cable. cable is a big waste of money and if not for the fact that I need my sports fix I would probably cancel my subscription. I got the bill the other day and it's close to 50 bucks now. but as I type out this entry I'm really enjoying epsn's "game of the millenium" mashup of the 70s steelers against the 80s niners so what can I do.

What's a mashup? It's when you combine different tools/data in order to come up with something cool. Think about overlaying craigslist housing data with google maps. That's a mashup. So the fantasy matchup between the two powerhouse football teams of the 70s and 80s is like a football mashup. We can call this tv 2.0 and tag it with the keywords "1980 forty niners" "1970 steelers" "fantasy matchup" "game of the millenium"

before I digress any further the impetus for this entry is to say that 24 rocks. Jack bauer is bad ass as usual and for the past two nights I've been glued to the set. What a way to hook new users..give them two back to back episodes on consecutive nights. One has to disregard reality to fully appreciate the show but once that leap of faith happens it's edge of the seat excitement.

The eye candy this season has got to be the first lady's assistant played by actress Sandrine Holt. I had to dig into the 24 site forum to find her but I guess that's not as complicated as Chloe running a "double regression binary filter" on some image that has been magnified 50 times in order to pinpoint a former eastern block army sergeant. but let's get this straight, she's just a bonus (and she could be a villain later on which would suck cuz I can't root for villains..pretty sure that'snot her character though)..I'm a fan of the show for the storyline first and Sandrine second.

back in san francisco

2006-01-03T16:55:00.000-08:00

we left vegas at 2pm and arrived in San Francisco at 3am. After dropping everyone off and returning home it was past 5 before I rolled into bed. Fortunately, I scheduled the entire week off so I was able to sleep in this morning. the most memorable moment of the trip was watching the countdown and the entire street erupt into celebration when the new year arrived. the streets were closed and we we on tropicana and las vegas blvd simultaneously checking out the fireworks coming from mgm and the roof of excalibur. the crowd wasn't too wild but there was the usual mix of drunks balanced by the ones of families with strollers. I'm surprised when I see both sets of people...especially the families with strollers crusing the strip so late at night.

gambling wise I probably spent an hour at the blackjack tables the entire 5 days. I also entered my first poker tournament and placed in the money. I finished 5/40 in a no limit game and my friend finished 2nd. The hand I busted out on was pokect 8's. I was up against 9-2 and the guy flopped a set and then had a quad on the turn. but that's poker...:) I was low stacked and my all in was just his big blind.

it was a pretty good trip overall and a nice way to start the new year with good friends.

vegas baby

2005-12-28T21:55:00.000-08:00

I just returned from the airport where I dropped some friends off to pick up the minivans. We're leaving around midnight so we should be there to enjoy a nice brunch. I'm planning on catching a show there...Kanye West is going to be singing and Celine's definitely an option if I'm willing to pay a scalper since the tickets are sold out. If all else fails I'll probably check out Lance Burton's magic show.

Spring MVC RedirectView appends jsessionid

2005-12-28T15:06:00.000-08:00

Back when I worked at Sun we used to have a shutdown period the last two weeks of the year. I'm not sure if they still have that policy but I wonder why all companies don't operate like that. This time of year most people are either on vacation or are in vacation mode while at work. In fact, most large projects are not considered or launched until the new year anyway.

I work at a pretty small shop and it's like that for us too. I had a day free before going on vacation so I volunteered to fix a couple of minor bugs. One of them dealt with removing a ;jsessionid=????? that was being appended to one of our links. If you're not familiar with what that is, it's basically tomcat's way of tracking sessions when a user does not have cookies enabled. Instead of storing it in a cookie, that string is passed around in the URL. But in order to see the jessionid string in the first place, it needs to be enabled somewhere in your code.

One method is using c:url tags in your jsp pages. Another is by using HttpServletResponse's encodeURL or encodeRedirectURL methods in your controller code. Usually it's a good idea to include these calls but in our webapp we do not store session data and that string was just adding extra noise since the page where it appeared is used as a launchpad to forward to a perl page.

My first intuition was to look for the c:out tag but no luck there so I searched for the encodeURL methods. Ack..no luck there either. Where could this be coming from? I then saw a call to spring mvc's RedirctView class so I figured it might be in there. The docs made no mention of jsessionid's so I looked at the source code. A ha...RedirectView actually calls encodeURL before the redirect so that was the culprit. Another one of the many benefits of open source...full source code view.

I described the problem to Andy and he showed me how to easily view a library's source code in IDEA. If you go to your preferences and then look at the libraries in your project, you can add an entry point to the src code on your local machine. The same goes for api's too. The best thing about it is that you can keep it in the zip file and IDEA will still find it.