Learn Things
[Recent Entries][Archive][Friends][User Info]
Below are the 20 most recent journal entries recorded in the "shack_a_nerd" journal:[<< Previous 20 entries]
|
10:09 pm
[Link] |
Online Banking, and it doesn't work
In lieu of being creative this evening I tried to at least be productive and set up Gnucash to automagically talk to my financial institutions using Direct Connect. There are two I really care about, HSBC for my checking account and CapitalOne for my credit card. Neither support Direct Connect "any more", although they seem to have in the past[*]. It seems the "preferred" mechanism these days is Web Connect, which entails logging in using your browser, selecting an "export" function, specifying a date range, saving a file to disk, and importing that file into your financial software. In other words, it's pretty damn far from being automatic. (AFAICS this is true even in commercial financial software such as Quicken or MS Money.)
But that can't make sense.. can it? Why would they just drop an obviously useful function like that?
Does anyone have any experience in this area? It looks like there are online services like Mint.com and Yodlee MoneyCenter which claim to connect to lots of financial institutions -- perhaps even these -- but I can't tell for sure with my primitive Google skills. Is Direct Connect a feature worth looking for in a financial institution? What's the right approach for a nerd with a bank account in 2009?
Ethan
[*] The story for CapitalOne is of course more complicated -- see msmoneycaponesupport.blogspot.com for the full story.
Tags: hacking, technical
|
|
11:11 pm
[Link] |
Incoming story
I have been working for a week or two on a new short story. The working title is "Salesmen" but really that's arbitrary. (Comes from the first line of the story.) I'm hoping to submit it to SHINE. Right now it's about 3000 usable words, most of which is crap. I've been writing almost entirely on my N810, and it's been an interesting experience. When you're clicking out sentences one letter at a time, you spend a lot of time with each sentence, and you find out that most of them are terrible -- warped reflections of the intents you had. The first draft isn't even done yet, and I've written at least 800 unusable words that veered off into the weeds very badly, so it's been bumpy going.
I'm writing this note for two reasons:
One, it would be cool if people I knew submitted stuff to SHINE too. It would be even cooler if they were published. I know at least one person reading this harbors vaguely literary ambitions. I hope this note will spur them to write, because hey, even Ethan's trying his hand at this crap. (Ordinarily I would wait until I was done, but I don't think that will give anyone else enough time.)
Two, I am realizing that the story doesn't really have "conflict". I am not sure how to feel about this. There are themes in the story. Things happen. Progress is made. Eventually I think a climax is reached. That all seems like enough to me -- maybe that's what "conflict" really is. Maybe not; secretly, in my heart of hearts, I think I'm just having fun writing about some characters I care about.
Ethan
Tags: life, writing
|
|
11:40 pm
[Link] |
Optimistic Science Fiction
I just saw a post about optimistic science fiction, including mention of a compilation due out called "Cheer Up, Universe", and one that's still soliciting stories called SHINE ("an anthology of optimistic near-future SF").
Those of you who have hung around me long enough have probably heard me mention Science Fiction Without the Future, an essay criticizing Science Fiction as a whole for tending towards darker stories of the future, musing that this might reflect a larger alienation with technology on the part of the Baby Boomers who still read Asimov's, etc. This essay really resonates with me, and as a card-carrying elitist I have no choice but to love the thing that isn't done as often. (Best new authors to produce optimistic sci fi: Cory Doctorow and Charles Stross.) Besides, I try to be an optimist in my day-to-day dealings, so it's kind of a foregone conclusion that I'd like to write a story for SHINE. I'd like it even more if people I knew wrote stories for SHINE. If you have a short story you've been waiting to write, now might be your chance. (Guidelines: "Convincing and optimistic: Imagine that we are the biggest skeptics on the planet, then show us how things can change for the better, and persuade us.")
Mom suggests that people of my generation don't really get our science fictional "jollies" from pulps/books any more, but instead from somewhere else. I think that's an interesting idea -- perhaps we draw that same "sensawunda" or sense of "progress" from reading blogs, buying new gadgets, lifehacking -- and P.S. also you can get them from tor.com, the best watering hole for science fiction discussion today. Art forms are changing all over -- as you may know from reading the Thoughtcrime Experiments anthology, "It’s well known that there’s an oversupply of stories relative to readers", and the repercussions are still being felt.
Ethan
Tags: memetics, writing
|
|
07:54 pm
[Link] |
G1 support for external keyboards
I got a few adapters to convert the mini-USB connector on my G1 to a USB-A female port (suitable for plugging in a keyboard or other peripheral. It doesn't work; my suspicion is that the phoe doesn't have an actual USB host chipset. Live and learn I guess.
I also grabbed a Bluetooth keyboard, which seems to pair with the phone, but doesn't actually send keystrokes. As far as I can tell, Android still doesn't support the keyboard Bluetooth device profile, even in cupcake.
Ethan
Tags: review, technical
|
|
11:59 pm
[Link] |
Want a pony?
It's a well-known meme that software developers will say "I want a pony" to indicate that a just-made request is unreasonable -- either theirs, or someone else's. My guess is that the meme is based on images of young girls unreasonably asking for a pony as a present (completely ignoring the cost to purchase or maintain a pony).
Geek culture, internet culture, and several other such associated cultures are sexist to varying degrees. But a prejudiced person typically does not borrow memes from the target of their prejudice. Where did this meme come from?
Relatedly, Slashdot has (as an April Fool's joke) changed color scheme from dark green to pink, and changed tagline from "News for Nerds" to "OMG Ponies". Response was generally pretty positive.
I'd also like to suggest that it seems that computer programmer nerds tend to joke a little more freely about their sexual orientation. No numbers, just offhand observation, and I'd love to be proven wrong (since it could be some kind of observer bias).
Ethan
Tags: life, memetics
|
|
09:49 pm
[Link] |
Flipping burgers again
Started my first day of school today. I played nice with the other kids (and the other teachers?). Didn't insult anyone, although I did almost. (I said PC, not Mac; how hard can that be to get right??) I shouldn't bitch -- they gave me a bar of chocolate (dark, by happy accident), plus the "Operations Team" (i.e. receptionists/paperworkers/general make-everything-work people) are very attractive [hah! Proof I'm getting back to normal]. One thing is that they kept saying things like "We're happy to have you join us" -- how can you be happy when you don't even know what I'm like yet?
Of course, starting a new job means I have things to bitch about w.r.t. OS X. Today's:
* No support for ext2/ext3?? How can you possibly justify this???
* This hasn't reached the level of furious rant yet, but I do think the way environment variables are handled on OS X is a little fucked. Individual shells run /usr/libexec/paths_helper or whatever, but applications that are run directly from the GUI get a completely different environment -- to wit, one with the wrong $PATH. Why?
* Terminal.app still sucks. Fucking god, you have Control, Option, AND Command! You'd think one of them would be an Alt key.
* No pageup/pagedown keys. No forward-delete. And a NO-BUTTON touchpad. A new high or something.
Ugh. That's all. Time for bed (like an hour ago).
Ethan
Tags: life, technical
|
|
08:26 pm
[Link] |
Reset my G1 again -- or did I?
Friday I dropped my phone on the sidewalk. Friday night I had contacted Asurion, who handles the warranty/insurance for the phone. I pay $6 a month so that I don't worry about dropping it, but it turns out there's a $130 deductible, so that's what I paid. I had my new phone Sunday or Monday, I can't recall. I had to send in my old one, of course, but I had taken a backup using nandroid before I did, so then it was just a matter of getting the data onto the new phone.
It turns out there are a couple of gotchas to pay attention to here. The restore is much more complicated than the backup. You can verify, if you like, that the restore worked by untarring data.tar -- it'll create a data directory. Received text messages are in data/data/com.android.providers.telephony/databases/mmssms.db, call log is in data/data/com.tmobile.myfaves/databases/contacts.db, etc. However, in order to successfully restore:
- The android SDK builds a mkyaffs2image which creates a different image from the mkyaffs2image-$ARCH precompiled in nandroid. I didn't spend a lot of time looking at the differences; I doubt they're significant, but to be sure, I ended up using the precompiled one.
- You have to untar the file AS ROOT, in order to get permissions right. If you do this wrong, you will have fascinating crashes from certain applications, wifi won't work, and so on. (If you have just restored and have weird misc failures that seem to go away when you erase userdata, this is your problem! Other miscellaneous failures I encountered like this: my accelerometer wouldn't work in Bubble Level application, ringtones got set to random values,...) You should be able to use fakeroot or something like it if running as root bothers you, though. Someone should submit patches..
- You have to be sure to run mkyaffs2image or mkyaffs2image-$ARCH on the right directory -- /tmp/-data-tmp/data instead of /tmp/-data-tmp, or /tmp/-system-tmp/system instead of /tmp/-system-tmp. If you get this wrong, your phone won't boot.
- fastboot flash userdata ~/path/to/data.img as discussed elsewhere.
- You may also have to rm /data/misc/akmd_set.txt to erase previous calibration of the accelerometer if it still isn't working.
So now I have my phone back, only missing three calls from my call log. Good enough for now.
It seems that the de-jure standard way to do backups is "in userspace", using something like MyBackup Pro. But having a rooted phone allows you to use "commodity software" like nandroid to backup everything, even including applications (which something like MyBackup Pro can't do). For this reason I would advise anyone to root their G1 the first thing they do. [Insert spiel here about how Google sucks for making a non-open source phone, pulling the wool over our eyes, etc.]
In other news, there is a lot of interesting activity going on in the Android community re: the production of illicit images. JesusFreke has a "brand name" for having been the first to produce rooted images, but now there's someone else (Haykuro) who seems to be producing a competing series of images based on Cupcake. ALL I WANT IS MY emacs, my python, my git, my bzr -- it's REALLY NOT SO HARD!
Ethan
[Edit: hmm, fakeroot should work too, right?]
[Edit: 10:13 AM the next morning: added the part about how you have to specify the right directory to mkyaffs2image, since I struggled with that for a while.]
[Edit: 11:39 PM: last thing about the accelerometer.]
Tags: life, technical
|
|
04:24 am
[Link] |
8-Bit Tarot
Why wasn't I informed?????
Ethan
Tags: found
|
|
07:32 pm
[Link] |
Singularity
I'm sure it's clear to you that I must still be unemployed, since the frequency of my postings is back to once-every-four-days. With luck this won't be true for long, but who can say?
Lately I have been taking an interest in "the long view". You may have heard about the Long Now Foundation; they have a Long Bets project. Today I'd like to write a few words about the Singularity. Background reading might consist of the Hob story arc on Dresden Codak, but then again that might only confuse you.
"The upheavals of the last twenty years have been unbelievable: the kingdoms of England, Holland and Spain have been transformed as fast as scenery in a theatre. When later generations come to read about our history they will think they are reading a romance, and not believe a word of it."
--Liselotte in a letter to Sophie, 10 June 1706 [quoted in The Confusion by Neal Stephenson, p.811]
One argument in favor of the Singularity is that the pace of technological and also sociological change seems to be increasing. After all, we now have the Internet, with all that entails: Craigslist kills newspaper classifieds, the blogosphere and Google News kill AP, open source software kills Microsoft, etc. ad infinitum. There are other changes too: netbooks are introduced; smartphone techonology has taken off -- hell, even mobile phones are a pretty big deal -- and others from your own field, I'm sure. A graph of paradigm shifts shows that they seem to be happening closer and closer together. But are changes accelerating, or are our perceptions skewed?
For example, consider an educated person's worldview in 1800. In the last 25 years, there have been revolutions aplenty. Something called democracy is making a big splash; elections are held in America, the Netherlands. Napoleon Bonaparte is rising to the top of the new Republic of France, which is invading Italy. A vaccine for smallpox is administered for the first time. The cotton gin has just been invented; so has the optical telegraph. Wouldn't things be new, fresh, and exciting? Wouldn't you predict a world just around the corner that was free from tyranny, disease, famine?
Developments are always happening, and I bet a list of "paradigm shifts" from 1800 would show a lot of events that we now think of as insignificant or minor. My bet is that progress isn't really increasing -- only that it seems that way, because the developments we understand best, care about most, and therefore list as most relevant, are the ones that just happened.
Professor Ingalls, when I took his class on networking in 2006, said something like "When I taught this class in 2004, I said IPv6 was Just Around The Corner.. and that's just as true now as it was then." I laughed out loud, because to me the forgone conclusion was: "And in 2008, it'll still be just as true." I'm not convinced that a new world is just around the corner. The data might show that, or we might just be compiling it wrong.
Ethan
[Edit, 22 April 1:13 AM: The essential point of this post, which I'm sure I didn't make clear, is this. Imagine you compiled a list of all the important social changes that have taken place throughout history. Now imagine that 70% of the changes on your list happened in the last 100 years, and about half of them happened in your lifetime. Would you conclude that the pace of social change was changing, or merely that you have a statistical bias in your data collection techniques?]
Tags: life, long now
|
|
04:22 pm
[Link] |
HTTP Caching
My preliminary testing suggests that if an HTTP response is cached, then as long as that response is cached, the browser won't even ask the server if anything's changed (no HEAD method, no conditional request, nothing). In other words: the cache can get arbitrarily stale and the browser will never know. Sometimes the user can "force" the browser to get a new version using their "refresh" command, but I've definitely had trouble lately with stale CSS files.
More information than you ever wanted to know about caching, which browsers do it, and under what circumstances, can be found here.
In theory, you should be able to invalidate an item in a user's browser cache by making them POST to that URL, but 1. this doesn't work in current versions of FF, but it should soon, and 2. this doesn't affect OTHER users who might need their browser caches invalidated too.
For a web application developer, this raises some questions. It seems as though you don't get an opportunity to tell the browser that its cache is invalid -- AFAICT conditional requests are only issued under certain rare circumstances, when the browser already thinks its cache is stale. So if you have volatile information, you have to disable caching, or severely constrain it (Expires: five minutes from now), or you can work around it entirely. At 10gen, the application server we worked on had code that would read HTML and rewrite <img> tags to include an ?lm= "freshness" parameter, thereby working around the browser cache using knowledge of the filesystem. This would be pretty trivial to do in Lockers, since all URL generation goes through one method (Node.url_to), but it seems like a hugely ham-handed technique, plus it leads to fugly URLs (but only for <img> tags).
I did a little poking around on Flickr and Picasa, and I think they both use a variant of this technique. A Picasa image preview might look like
<img src="http://lh4.ggpht.com/_p3ZL1muhkaU/SeZKiTZ0YhI/AAAAAAAAABI/SUu5t8M7Klk/s720/DSCN0397.JPG" />
I have bolded the parts that change when the image changes (i.e. I rotated the image, and img tags changed). More of the URL might change too, if I played with it more. These tags therefore seem to be generated on-the-fly when rendering a photo gallery, based on information in the database. If you try to fetch one directly, you get an ETag like "v12" or "v17" and an Expires: about a day into the future.
Ethan
Tags: hacking, technical
|
|
12:54 am
[Link] |
Secret Project: delays FTW
I originally expected that I could save some time by implementing a shitty file uploader for the here-and-now in Lockers and it would be a fine stopgap until someone "did things right" (which meant writing an F-Spot plugin that would upload tags and shit). So I implemented an upload handler that would accept a zip file, git-add it, and then unpack it, and git-add all the unpacked files, and turn that bitch into a photo album. It turns out this is a REMARKABLY STUPID idea because although I may want a user to download a zipped version of a photo album, I'll probably want to generate it on the fly, and memoize it (which I do and works great) so there's really no use for the original zipped file WITH THE ADDITIONAL overhead of sticking that bitch into git. That's approx a 4x overhead for each image uploaded, when it really should be 2x. Not to mention that git-add mmap()s the file, which causes DreamHost to gun git ("you're using too much memory") once you cross approx 80 MB. Gunned git processes leave dead locks and so on. It's a bad scene. Additionally this means re-uploading the file, which at 400K/s upstream takes FOR-GODDAMN-EVER.
This dovetails nicely with the limit on FCGI processes running for longer than about 10 minutes without getting gunned.
This is just one of about four MAJOR, CRITICAL SHORTCOMINGS that I really should have addressed before trying to put Lockers into production. Instead now I am thinking I should spend another couple days polishing the thing, until I have at least ONE of the following:
- a multi-file album uploader, or an image upload API, or, god, ANYTHING
- ETags
- ?lm=foo on CSS/JS files so that I'm not stuck with weird caching failures when I want to make my layout a little less dysfunctional
- a link to view an image fullsize!
Of course, this doesn't even cover the MAJOR ESSENTIAL missing features like tags, user registration, a layout that doesn't make you want to eat kittens, and I'm sure you get the picture.
MAINLY this post is an apology to Beekey, who has been waiting patiently for me to upload photos from a couple weekends ago. Well, I'm not there yet. But I HAVE uploaded approx 750 MB of photos from roughly 2008-05 to 2008-06. Please feel free to visit them until I have found the patience to wipe the DB and start over.
Ethan
[Edit, 2:02 AM: and don't actually TRY to download a zip of any of the albums, because that process will get gunned after 5 minutes like the rest. Thanks for nothing, Dreamhost! I have no idea how I'm going to fix THAT..]
Tags: hacking, life
|
|
12:03 am
[Link] |
Rails plugins
It turns out you don't have to do anything in particular to "activate" a Rails plugin -- but you do have to restart WEBrick.
Ethan
Tags: hacking, life
|
|
11:56 am
[Link] |
Printing using Samba and CUPS
It turns out CUPS can print to a SMB-hosted printer using smbspool. It worked for me pretty much out of the box. First:
- Configure CUPS. On Debian the easiest way is probably to point a browser at http://localhost:631/
- Add printer. Name, Location, Description are up to you.
- Device: select "Windows Printer via SAMBA"
- Device URI: something like smb://machine-name/printer-name. Spaces are OK.
- Providing a PPD file: you actually have to do this, even for an SMB printer! smbspool fills in the role of lpr or equivalent -- a "dumb" transport to get data to the printer. Fortunately the CUPS database is pretty thorough these days, and if you get it wrong, you'll know.
Once the printer is set up using CUPS, Firefox et al will see it automagically. All in all, I got things working in like 15 minutes. Pretty impressive for what was once a huge weakness in the Linux platform.
Ethan
Tags: hacking, life
|
|
11:28 pm
[Link] |
More CSS nonsense
The rant has come and gone. What remains is information. Did you know about CSS's display: inline-block? It's really really helpful if you're trying to do a fluid grid layout, as might be done in an image gallery. You must be strong and ignore the float: left; approach to fluid grids. Instead try a cross-browser inline-block.
Ethan
Tags: hacking, life
|
12:21 am
[Link] |
Overthinking the obvious
Speaking of my secret project, I've been working on pagination. Let's say there are 48 things you want to look at. By default they are displayed 10 per page. You're on page 3 -- that is, you've already viewed things number 1-20, and you're about to view 21-30. Now you click "show 15 things per page".
There's two ways software can handle this, and they both suck.
1. It can say, "OK, now there's 15 items per page. Item 21, that you were about to view, is on page 2, so now you go to page 2." But page 2 has items 15-30, so you get to re-view items 15-20. In other words, this option is to maintain that the world is ordered according to pages and do the best you can.
2. It can say, "Well, you already viewed items 1-20, so now we'll just display 21-35." This option, as opposed to the "page" option, is "skip" based -- a request to view items indicates how many to skip. Its complications are a little more interesting: Which page is the user on when they view 21-35? How many pages are there? Which elements are showed on page 1?
I thought I'd look at how Google does it and maybe try to emulate that. I think their solution might be imperfect, but given the nature of the trade-offs, I'm not sure. Here's what I've found.
Google uses a skip-based solution -- you can tell by clicking the page numbers on a query like this one. It'll take you to a page with skip=10 or something. Page 1 is skip=0, page 2 is skip=10, etc. If you're on page 2 (with skip=10) and change your preferences to display 20 results, you'll be whisked back to skip=10 and see results 11-30. Interestingly, this is listed as page 2. [This leaves you the option to go to page 1 and see results up to 11, so I'm pretty sure this is the right thing to do.] Even more interestingly, page 3 will take you to skip=40, which means you'll never see results 31-40. You could also click "Next", which will take you to skip=30 -- which will show as page 3 (of 3) with a bonus next button!
This behavior may seem a bit insane, but it makes sense if you consider how many pages there are at each step of the process.
At the start, when you're at skip=0, if there are 54 results, then there are clearly 6 pages: 1-10, 11-20, 21-30, 31-40, 41-50, and 51-54. When you're at skip=10, there are still 6 pages, but you're on page 2. But when you change the number of results per-page to 20, now you're looking at 11-30. If you call this page 2, then page 3 is 31-50, and page 4 is 51-54.
But wait. 4 pages, with 20 results a page? That's unusual because if you start at skip=0, then you only end up with three pages: 1-20, 21-40, and 41-54.
So your pagination routine either shows an inconsistent number of pages, or you make it difficult for users to see all the results. I think Google is counting on users clicking "Next" when they're reading results sequentially instead of trying to access pages of results directly. If they do access results directly, they get the same set of results (21-40 on page 2) regardless of how they got there.
Frankly I feel like having an inconsistent number of pages is better than accidentally skipping results when a user clicks "3" instead of "next". On the other hand, I hardly ever click the page number myself -- I almost always click "next" and use the browser's back-forward to navigate. The only time I think I've used the direct-to-page links were when I was trying to get to the end of a result set to see how big it is. But Google's probably put a lot of research into how they paginate, because pagination is important to them; I don't think it's a good investment to re-research it; I'll just copy what they do.
Ethan
[Edit, 11:35 AM the next morning: whoops, screwed up the link to the Google search.]
Tags: hacking
|
|
03:52 pm
[Link] |
My secret project
Threw together a little demo of my latest secret project.
Ethan
Tags: hacking, life
|
12:15 am
[Link] |
CSS isn't complete crap
I was getting all fired up to write yet another rant about CSS and how much it sucks, but it turns out the vertical-align property works fine on elements with display: table-cell, and that's exactly what I wanted. The only trick appears to be making it work in IE.
Ethan
Tags: hacking, life
|
|
08:35 pm
[Link] |
Free software icons
Just spent a few hours looking through icon packages for KDE and GNOME, because my madness is many and variegated. There's a lot of them. They match. It's kind of beyond me to make a single thing that looks nice, but to make a bunch, that look nice in the same way? That's pretty amazing. It must be nice to be an artist.
How I stumbled into this: I'm using random GPLed icons to fill out some interface elements in my secret project. It turns out SVG is really cool for this because you can generate icons of whatever size using them (but sometimes they won't look right). But it seems like picking an icon here, an icon there, will lead you astray pretty quickly. But no set contains icons that "say" all the right things, or say them the way I want them to be said.
Ethan
Tags: hacking, life
|
|
02:54 am
[Link] |
2009
Around this time of night, the taxis at Flatbush Avenue honk at subway passengers just emerging from the underground. It didn't used to always be like this; it's a recent -- last year or two -- innovation. Frankly it drives me a little infuriated, because it's a blatant infringement on my personal space and time as I take this fifteen minute walk. How many passengers do they really get that way? Do they really all have to hoot at once? What about the poor bastards who are trying to sleep (at the Junction -- but still)?
But on the other hand there's something organic about it, something relentlessly and irredeemably human about that ceaseless staccato blaring noise. And that's not all bad.
Feeling like I'm drunk-dialing the Internet, except sober and with a keyboard. Ahh, technology. The words that emerge from my mouth lately have been: "I dreamt there was something I cared about" -- which is completely misleading, but when the subject is words that won't get out of your mind, it hardly matters whether they're accurate, isn't it so?
Ethan
Tags: experimental, life, writing
|
|
10:11 pm
[Link] |
Line width
My eyes almost popped out of my head when I read this on the Emacs Wiki:
Yet another reason is that narrow text is easier to read, as it requires fewer and shorter saccades per line. -- SeanO
Read that over. Take as many saccades as you like. Fewer saccades per line? What a bogus metric! How about the fact that there are more lines?!
I've seen this meme bandied about a few times before, particularly by Paul Graham, but never justified. My money is on "no significant difference", similar to the question of whether fonts with serif or sans serif are more legible.
Here's a survey I found about the optimal line length for reading prose text from a monitor. This article concludes that longer line lengths are "better" (as determined by speed of reading), in contrast with the findings of early typography research (e.g. Tinker and Paterson, 1929) that suggests a line length of three to three and a half inches. I briefly tried looking for the actual Tinker/Paterson article but came up blank.
Ethan
Tags: hacking
|
[<< Previous 20 entries] |
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
shack_a_nerd's journal