Never Clever

“What people forget is a journey to nowhere starts with a single step, too.”

Week 127

Mar 12

Posted by admin in Code, Work | No Comments

(idea cribbed from BERG, week count done by Wolfram Alpha because I am too lazy)

"[T]he beauty of reading a page of de Selby is that it leads one inescapably to the happy conclusion that one is not, of all nincompoops, the greatest."
The Third Policeman*

Mix of a week. Started with some edits to the XML parser I wrote for Financial Institution Client as part of a project that decrypts (GPG) a large set of XML documents, skims them for the parts we're interested in, dumps the relevant bit into an Expression Engine database and then transforms all that via XSL to display a customizable dashboard (jQuery UI) on the front end. How's that for a keyword-rich blog post? Tangentially speaking, this week reminded me that of all the languages, technologies, whatever you like that I work in, XSL is probably the easiest one to make a great mess in. It looks just like XML and HTML, how hard could it be? A little bit of XPath knowledge and you're good to go. To make a complete mess. When you write an infinite loop in a normal language, it's pretty obvious: you sit there a while, the computer starts to get noisy, the lights go dim. After a few dozen times, you realize what you've done. XSL is (like) a functional programming language. And with all that recursion, it's easy to make a computer do something Big Number of times. Just harder to spot.

Also did some final-mile edits on a social network application from Slim Kiwi built on top of Pinax (and Django).  Hoping it's truly "final-mile" as the project is really cool and I'm quite proud of it. There's a ton of geolocation and mapping going on and a fair number of other bright things happening (the bright ideas being supplied by other team members and the bright implementation by Django, obviously).  I was able to roll some of the geo search and Google Maps integration right into a site for Community Trust Bank, a project from Lightfin Studios (you can see the location stuff at the branch & ATM finder). That's the second bank site I've built with Lightfin, the other being the much-closer-to-home (but harder to spell) Piscataqua Bank (built in .NET) which got some updates this week as well.

At the end of last week I started to ramp up on my second Django site with Lightfin. Not much to say about it yet except that it integrates with a third-party API which reminds me of one thing: I hate SOAP (the overly-verbose web service format, not the cleaning product so beloved by my ancestors they enslaved a leprechaun to endorse it). Please don't ever use it. I'd rather parse faxes by hand. If you're stuck dealing with SOAP in Python, Suds seems to be the best parsing package out there. I'm sure there was other stuff going on, but the only thing I can think of is some cleanup I did of an old ASP site and the less said, the better.

* Indie/ hipster required disclaimer: I am re-reading The Third Policeman, I read it well before it showed up in Lost, so cram it.

Tags:

On Optimization

Feb 25

Posted by admin in Code | No Comments

It's strange what you can get used to: the current social network site I'm working on has a page with 216 database queries on it. Used to be I'd get the hives if I hit a dozen queries on a page.

"216! Did you know databases let you bring back more than one row at a time nowadays?"

Yes. The project is in Django (and built on top of Pinax), so it's the ORM making all those queries, not me. It's one of those social network site pages that aggregates activity from everyone you follow. It also shows details about them, how far they are away from you and any comments on the item, so there's only so small I can make it while coloring inside the lines of the mapping system. I've already fallen back to raw SQL for one of the elements (there are a couple of places, and sure to be more in the future, where we return a list of the database ids of all your friends so we can use them as part of " AND id in (x, y, z)" queries. Doing that through Django resulted in one query to the database for every friend you have. Given this was causing a slowdown when I'm the only user of the site and I only have 3 friends (one is another tester and the other two are dogs I know, so it's kind of a "Bob" situation (specifically the dog part and not the rest)), I had a suspicion that wasn't going to scale. Modified that, added some caching, got smarter about some lookups (I thought I'd only hit the db once no matter how many times I referred to a model's property in a function) and things are back to running smoothly.

"216!"

Hey, it was 1066 when I started a day ago. Or something close to that. I've got 1066 on the brain because I've been thinking about William of Orange and before you say--

"Write code for a job and think about William of Orange in your spare time. You must be a hit with the ladies."

--that, let me point out it was in reference to a Dutch Oven joke. That has to count for something.

"Undoubtedly. Perhaps 'lady killer' is more literal than figurative in your case."

Regardless, given the nature of the screen, aggregating a dozen types of activities from an arbitrary number of users, I don't think the current solution is the long-term answer, so I buttoned it up as best I could.

"As best you could? Implement the long-term solution now."

That would be solving a problem I don't have (c.f., "premature optimization", "YAGNI"). Given the data for this screen is derived from other objects in the system anyway, I think the long-term solution is to move this data into a nosql store (here's an example of using CouchDB in Django now and future updates to Django should improve support for this kind of thing). It's important to remember traffic issues fall under the title Good Problems to Have. While I'd love to spend a couple of days implementing this rightnowyespleasecani, if the overall project never takes off, it would be unfair to ask the client to pay for something they didn't ask for and never needed.

"216!"

I'm already obsessing over it on my own. Why do you think you're here?

Tags:

Microformat Proposal: Coding Experience

Sep 19

Posted by admin in Code, Work | 1 Comment

When I'm working, even in a language I know well, I often search for how to do something; either because I don't know or because I feel there's a better way (as @ed_atwell says, "I don't know, but I bet my friends Larry and Sergei do). My personal system for filtering code search results looks something like:

  1. Blogs I trust
  2. Personal blogs
  3. Development sites (e.g., 4guysfromrolla.com, etc.)
  4. Mailing lists and newsgroups1
  5. Forums
  6. Expert Sexchange

Regardless of where it comes from, there's no way to know if it's right. It's human nature to use the first thing that works (if under deadline, even the first thing that kinda works will do). As Jeff Atwood has pointed out (twice) , the danger is you might be copying off the paper of someone dumber than you2. Because of this, I'd like to propose a microformat (assuming one doesn't already exist, given I didn't bother to check with Larry and Sergei) to indicate an author's experience with a language.

Immediate disclaimer: I realize this is a programming solution to a human nature problem and those never work, but bear with me, because my hope isn't to fix the problem, but to provide some metadata that will let machines do the work for us so we can stay lazy. Given that is in line with Newton's First Law, this will obviously be a huge success.

The format doesn't need to be very complicated. In fact, I'd prefer if it just provided a few bits of raw data that could be remixed by search engines however they see best. The data provided would stay the same but the algorithms could be tweaked for better results (though that would require feedback), providing an incentive for search engines to consume the format. Make the data something rough, broad and quick to fill out, like years of experience with the language and a simple measure of number of lines written (e.g., none, 10, 100, 1,000, 10,000, a whole bunch). There are any number of issues with using Lines of Code (LoC) as a metric (mainly that an idiot can say in 1,000 lines what a smarter person can say in 10), but if the ranges are broad enough, it should dampen the effect.

Bolt this format onto syntax highlighting engines; this blog, for example, uses WP-Syntax to format the few, poor code samples I provide— one more panel in the plugin admin that allowed me to store a hash of [language name, years, lines of code] would allow the plugin to provide that information in any page using the languages and output a visible box on the page so inexperienced users who come to the page and see my code could know it was terrible without knowing it was terrible. Add it into the syntax formatters for popular forum software (and allow users to specify their experience) and every code argument in a forum post becomes a little easier to follow.

The format doesn't tell you if a snippet is correct, it just gives you some background information (assuming the author is honest in their self-reporting). The danger would be users trusting a snippet blindly because the author has 10 (bad) years of experience (a sort of "Appeal to authority") while better code from "newer" users goes ignored. That's a human nature problem and obviously you can't solve those with programming (/broad wink).

1. I'd rank these higher, especially official groups for languages and systems except for two reasons:

  1. They tend to be so ill-formatted and the ability to follow threads varies wildly from site to site
  2. The advice can be good but dated: it's easy to find perfectly legitimate Python answers from 2000 or so. While the answer is fine, it's possible there's a newer idiom and in a language like Python, where there's "one right way", the right way will be the way that the language has been optimized to work.

2. Basically unrelated story that I've crammed in because I always tell it because it cracks me up: in high school, we had to go to the local public high to take the SATs. The person sitting next to me scribbled furiously throughout the test and was always the first one finished (which frustrated me to no end). When we were walking out, he turned to us and said, "Dude, I just made pretty pictures with the bubbles."

Cheating at The Beatles: Rock Band

Sep 2

Posted by admin in Video Games | 1 Comment

My favorite thing about the game is the harmonizing and the way it increases the feeling that you're really in a band, but if you're all about the score (or bereft of friends), feel free to take advantage of these two pieces of information:

  1. The different singers do not have to sing different parts
  2. The multiple scores are based solely on having multiple microphones

In this case, 1 + 2 equals, "If you stick three microphones in front of your face and start wailing, you'll be credited as three singers, including the double and triple bonuses". Kids, you're only cheating yourselves.

Django/ Pinax: Problems With Login() in Unit Tests

Sep 2

Posted by admin in Code, Work | No Comments

This is the first in what promise to be a number of "Stupid Django Tricks" where the "stupid" is me and not Django. I was having a good deal of trouble creating unit tests for authenticated views (i.e., pages that require a user to be logged in) for the Pinax project I've been working on. I dug up two problems, one of which is on Pinax and one that's entirely on me:

  1. Pinax's settings.py file does not provide a setting for AUTHENTICATION_BACKENDS, so the test client's login method doesn't know how to log your user in. Specify "AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend',)" in your settings file. Actually, I lied. That's the default value for the setting; having gone back and re-run my tests without it specified, everything works, which means the only idiot here is the guy who . . .
  2. Don't create users by specifying the password directly in the declaration (e.g., user = User(username='Dummy', password='goodluck')). Use the set_password() User method to properly set the password.

I've run into a fair number of issues working in Django where Google wasn't helpful. I think 90% of those issues were because no one else was dumb enough to make such an obvious mistake. The other 10% were typos.

Monty Back, Rommel Still Dead

Jul 30

Posted by admin in Sports | No Comments

Bob Montgomery is doing the color for today's Red Sox game and I can't figure out how to feel about it. Monty and the late Ned Martin were the voice of the Red Sox (on WSBK TV38) when I was growing up and it's strangely transporting to hear him again. He's done some Pawsox games, but it's hearing him back, like Jake Taylor (a fellow catcher) getting one last chance with the parent club. And it's like he's never left: same dulcet tones, knows the team, not a sign of age (unless you get a look at the tombstone of a gut he's developed in retirement).

It wasn't until about year 3 of the Don Orsillo Experience that I realized I'd seriously undervalued Sean McDonough. Orsillo is fine, but he's a generic Connecticut School of Broadcasting voice. Close your eyes and he could be talking about the Kansas City Royals. Sean McDonough's only sin for me (beyond the too-goofy adulation of Remy) was not being Ned Martin. Hearing Monty makes me feel like I'm ten, I've got a whole summer in front of me and there's nothing to worry about for the foreseeable future*. And that shit will get you killed.

* Of course, there were no World Series wins back then either

Free to a Harmonix Home, Rock Band Idea

Jun 2

Posted by admin in Video Games | 1 Comment

Why doesn't Rock Band allow you to create additional cities and venues? Nothing fancy, just the ability to set a city name and country, then create some venues. Venues would just let you select from the existing arenas and clubs1 (the 3D animation tool for user-created venues feels like more of an RB3 thing). It seems like an obvious idea for selling more content: allow for users to download cities from other users or Harmonix, but require them to have x downloaded songs to be able to use the city. Maybe the venue creation could have a recommended genre for what types of songs to choose from a user's collection, but not require specific songs.

Except in one case: if a label wanted to set up a "city" that contained historic venues a group played at on their rise to stardom and require you to buy various tracks to use them, that seems like a really cool way for labels to increase artists' sales in Rock Band (or Guitar Hero). It'd be like a low-cost version of Rock Band: Beatles for any group that cared to take the time/ money to get the venues created.

1. I am, of course, ignoring the legal issues that could arise from letting people create venues with names like "This place in my hometown sucks b@!!s", but it's my post and I'll do so if I want to.

YUI Rich Text Editor in Django Admin

Apr 30

Posted by admin in Work | 4 Comments

This ain't exactly rocket science, but it took me an embarrassing amount of time to get there, so I'm posting the code for next time. This will turn a given textarea in your admin area into a WYSIWYG. It's got a fairly small feature set, but that's only because I've stripped most of them out. You can add them back in by taking a look at the documentation. Per the Django docs, create an admin folder under one of your templates directories, then add subfolders for the app and model (though you can do just one if you want it to apply to all forms in the app or do neither to apply to all apps and models in your site) and add this as "change_form.html" (it took me an extra 10 minutes to get this done because I was sure it should be named "change_form.py" in spite of copious amounts of documentation that said otherwise):

 
{% extends "admin/change_form.html" %}
 
{% block extrahead %}{{ block.super }}
<!-- Skin CSS file -->
<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.5.1/build/assets/skins/sam/skin.css">
<!-- Utility Dependencies -->
<script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/yahoo-dom-event/yahoo-dom-event.js"></script>
<script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/element/element-beta-min.js"></script>
<!-- Needed for Menus, Buttons and Overlays used in the Toolbar -->
<script src="http://yui.yahooapis.com/2.5.1/build/container/container_core-min.js"></script>
<script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/menu/menu-min.js"></script>
<script type="text/javascript" src="http://yui.yahooapis.com/2.5.1/build/button/button-min.js"></script>
<!-- Source file for Rich Text Editor-->
<script src="http://yui.yahooapis.com/2.5.1/build/editor/editor-beta-min.js"></script>
<script type="text/javascript">
		// YUI editor
		var editor = new YAHOO.widget.Editor("id_content", {
			handleSubmit: true,
			toolbar: {
        buttonType: 'advanced',
        buttons: [
            { group: 'fontstyle', label: 'Font',
                buttons: [
                    { type: 'select', label: 'Arial', value: 'fontname',
disabled: true,
                        menu: [
                            { text: 'Arial', checked: true },
                            { text: 'Verdana' }
                        ]
                    },
                    { type: 'spin', label: '10', value: 'fontsize', range: [
10, 16 ], disabled: true },
					{ type: 'color', label: 'Font Color', value: 'forecolor', disabled: true }
                ]
            },
            { type: 'separator' },
            { group: 'textstyle', label: 'Font Style',
                buttons: [
                    { type: 'push', label: 'Bold CTRL + SHIFT + B', value: 'bold' },
                    { type: 'push', label: 'Italic CTRL + SHIFT + I', value: 'italic' },
					{ type: 'push', label: 'Underline CTRL + SHIFT + U', value: 'underline' }
                ]
            },
			{ type: 'separator' },
            { group: 'indentlist', label: 'Indenting and Lists',
                buttons: [
                    { type: 'push', label: 'Indent', value: 'indent',
disabled: true },
                    { type: 'push', label: 'Outdent', value: 'outdent',
disabled: true },
                    { type: 'push', label: 'Create a Bulleted List', value:
'insertunorderedlist' },
                    { type: 'push', label: 'Create a Numbered List', value:
'insertorderedlist' }
                ]
            },
            { type: 'separator' },
            { group: 'insertitem', label: 'Link',
                buttons: [
                    { type: 'push', label: 'HTML Link CTRL + SHIFT + L',
value: 'createlink', disabled: true }
                ]
            }
        ]
}
			}
			);
		editor._defaultToolbar.buttonType = 'advanced';
		editor.render();
</script>
{% endblock %}
 
{% block bodyclass %}{{ block.super }} yui-skin-sam{% endblock %}
 

Like I said, not rocket science. It adds some CSS & JavaScript includes (which are remotely-hosted, so you don't even have to worry about media roots or how it works locally vs. live) and then a bit to add a class to the body tag for the YUI skin.

Tags:

My Name is Crew

Apr 4

Posted by admin in Uncategorized | 3 Comments

I am posting this because I need to keep track of these things. Dreamt last night Michelle and I put on a community event centered around the retirement of some guy that had been a social worker all his life. After the dance crew came off the stage, there was a PowerPoint presentation of his life that I put together (each slide featured an allegorical photo of a raven) and a country music song. All I remember is:

My name is Crew
My name is Crew
Saving kids is kinda what I do
[a capella]Leading them away from a path of self-destruction . . .

And so on. The ravens were a result of watching a David Attenborough documentary last night and Michelle points out "kinda what I do" is a phrase that Bill Burr repeated in the stand-up show we watched again last night. So that explains a bit of it, but I still don't get where these dreams with original music come from. I must be choking off my creative brain during waking hours. Earlier this week I'd dreamt my friend had walked into a convenience store and declaimed a filthy sonnet in perfect ABAB rhyme scheme explaining why he needed to buy the New York Times Sunday Magazine and not the whole paper.

Tags:

Expression Engine if Clauses

Mar 16

Posted by admin in Code, Work | No Comments

This is the kind of thing that's not worth a blog post except some day it might save one person hours of frustration. Expression Engine apparently doesn't like it when if statements either span multiple lines or when the trailing curly brace is pushed to a new line. I can't quite run down which it is, but it's not all that important: if your if clause isn't behaving as expected, make sure it's all on one line without any extraneous whitespace.