Up to now, Ballpark Genius was entirely a look-at-the-past site, projections and season stats, nothing about the game happening right now. Added a whole vertical for that: today.service polls the MLB Stats API every 5 minutes, imports today’s schedule, and for any game that’s gone Final since the last poll, pulls the boxscore and writes the batting/pitching lines. The MCP tool for boxscores hands back formatted text instead of JSON, so those get parsed and upserted under a synthesized retrosheet-style id, MLB{gamePk}, since MLB doesn’t have one yet for a game that just ended.

Four endpoints under /api/today, each cached 5 minutes, and search got a new intent detector, detectTodayIntent(), that recognizes four shapes of question: what games are on today, who’s leading today, how’d a specific player do today, how’d a specific team do today. It slots into the existing semantic search response as an optional field rather than a separate code path, so “what’s the score of the Yankees game” and “who’s on pace for 40 home runs” both come out of the same box.

On the frontend, a TodayBand widget on the homepage, a game rail plus tabs for home runs, strikeouts, and total bases, refetching itself every 5 minutes via react-query. It also seeds the plumbing for live in-progress linescores, which isn’t built yet, GameCardLive already accepts a linescore prop, there’s just nothing populating it. That’s its own ticket. Shipping the “today” layer without the “live, mid-game” layer felt like the right cut. One is a poll every 5 minutes against data MLB has already finalized. The other is a websocket-shaped problem I didn’t want to solve in the same afternoon.

The Ballpark Genius CLI has a Levenshtein-based suggestion dropdown, type part of a command, it guesses the rest. Escape was supposed to back out of whatever you’re doing. Instead it quit the whole program the moment the dropdown wasn’t visible, even if you were mid-sentence typing something with real text already in the input box. You’d tap Escape out of habit, meaning “close this dropdown,” and the CLI would just leave.

Fixed it as a priority chain instead of one flat check: if the dropdown’s showing, close the dropdown. If it’s not, but there’s text in the input, clear the input. Only quit the program if neither applies, dropdown hidden and input empty. Escape now does the smallest reasonable thing first and only exits as a last resort, which is what everyone means by “hit escape” in literally every other program with a text field.

For a stretch, Ballpark Genius’s home run leaderboard had Jonah Heim on top. One home run. The query behind it filtered WHERE team_id = 0 to get season-total rows instead of per-team splits, correct for a player who’s been traded, MLB gives those a team_id = 0 aggregate row, and wrong for literally everyone else, who only has per-team rows. That left about seven hitters and eight pitchers as the entire eligible pool for every leaderboard on the site. Whoever had the highest single stat among that tiny, mostly-irrelevant group won.

Fixed with one query instead of two: DISTINCT ON (player_id) ORDER BY player_id, team_id ASC picks the team_id = 0 row for anyone who has one and falls through to their single per-team row otherwise, then the outer query sorts and takes the top N like it always should have. Checked it live after: Judge, Alvarez, Schwarber, back where they belong.

Small, dumb, expensive bug. The kind where the fix is three lines and the entire distance between “leaderboard” and “not a leaderboard” was one wrong WHERE clause the whole time.

Ballpark Genius has a champion/challenger loop that tunes a new prediction model every night and promotes it if it’s actually better. It promoted heuristic_v9 over the baseline weeks ago. The live API kept using the baseline’s math anyway, just with a v9 label stapled to it.

generateGamePrediction() had the weights hardcoded as literals, eraSwingMax, parkRunFactorWeight, homeAdvantageBase, the works, regardless of which model version the database said was champion. The tuner could promote a model with meaningfully different weights and the prediction endpoint would go on doing exactly what it had always done. Self-improving in name only.

Fix was to actually read the champion: getCurrentChampion() loads its weights once at the top of generateGamePrediction, and calculateWinProbability takes them as a parameter instead of reaching for a constant. Checked it against a real game, Nationals at Marlins on May 10th: 38% home win probability under v9, 46% under v0, an 8-point swing from wiring alone, no new data, no new model, just finally using the one that had already won. Across that week’s slate of 92 games, v9 picks the home team 47% of the time vs v0’s 65%. The old hardcoded weights had a structural home-field bias that was never supposed to survive a real promotion.

Somewhere on this project there’s a lesson about verifying the thing you shipped is the thing that’s running. I already knew that lesson. Here we are anyway.

I took 30m to experiment with Node v20 yesterday, initially focused on experimental permissions. I noticed features that have been introduced since Node 14 that apply to Sports Card Investor’s engineering org, e.g. corepack.

Node/CorePack

One of those is Node’s corepack, which holds pnpm as an add’l official package manager that it will manage for you. You enable corepack like so. You declare which package manager and version thereof your Node project is using like so

Node::Test

Another is the introduction of the Node-native test package, which works really nicely with assert. assert has always been useful for Node projects but the introduction of test makes it so that jest is just not necessary for many projects

Experimental Permissions

Node 20 now has a method for locking down which system aspects your code can access & have side effects on. I’m predicting that this feature will also be useful for debugging issues related to third-party code interacting with the file system (e.g. “what the hell is mutating this file?”). Here is an example of making FS permissions read-only and limited to the current directory & its descendents. Experimental Node API usage triggers warning messages by default, I’ve also used an environment variable to silence those—the point of that repo is using experimental APIs

Tying it all together

By running these tests with pnpm I’m combining corepack, test & --experimental-permissions to verify that FS reads are enabled in the project directory, while FS writes are not

An example of running cluster on Node 14, including w/ ESM & semantic updates for ES2020

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import cluster from 'cluster';
import {createServer} from 'http';
import {cpus} from 'os';

const isMainCluster = cluster.isMaster;
const {length:CPUCount} = cpus();

if (isMainCluster) {
  console.log(`Main ${process.pid} is running`);

  // Fork workers.
  for (let currentCPU = 0; currentCPU < CPUCount; currentCPU++)
    cluster.fork();

  cluster.on('exit', worker =>
    console.log(`worker ${worker.process.pid} died`));
} else {
  // Workers can share any TCP connection
  // In this case it is an HTTP server
  createServer((req, res) => {
    res.writeHead(200);
    res.end('hello world\n');
  }).listen(8000);

  console.log(`Worker ${process.pid} started`);
}

Changes from the current Node.js ‘cluster’ docs include:

1. Using import over require
2. Using named imports over the default where feasible
3. Using clearer, more semantic identifier names, e.g. currentCPU v. i
4. Replacing archaic terminology, e.g. Master w/ Main

Output w/ ESM Node code w/ syntactical & semantic updates

1
2
3
4
5
6
7
8
9
10
11
12
13
14

> $ node index.mjs                                                                       ⬡ 14.15.1 [±main ●]
Main 57290 is running
Worker 57292 started
Worker 57295 started
Worker 57291 started
Worker 57294 started
Worker 57293 started
Worker 57296 started
Worker 57298 started
Worker 57297 started
Worker 57299 started
Worker 57300 started
Worker 57302 started
Worker 57301 started

See here for the full repo

One of the constants in my software dev career has been a predilection for reducing redundant code as much as feasible. There’s a JSX-fluent CLI tool that been helping me do that at iTRVL.

Running npx jsinspect ./packages/agent/src/components/Itinerary/EditItinerary/*Step.js will give you a set outputs like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

./packages/agent/src/components/Itinerary/EditItinerary/FlightStep.js:132,136
<EditableInput
  textValue={formatMoney(step.sell, 'USD')}
  inputValue={step.sell}
  editOnly={true}
  inputProps={{ className: classes.editableAlignRight }}

./packages/agent/src/components/Itinerary/EditItinerary/RoadStep.js:136,140
<EditableInput
  textValue={formatMoney(step.sell, 'USD')}
  inputValue={step.sell}
  editOnly={true}
  inputProps={{ className: classes.editableAlignRight }}

------------------------------------------------------------

Match - 2 instances

./packages/agent/src/components/Itinerary/EditItinerary/RoadStep.js:94,97
<Paper className="stepLine" elevation={0} style={{ marginLeft: '0px' }}>
  <Box className="header">
    <Box className="type-icon">
      <AirportShuttle />

./packages/agent/src/components/Itinerary/EditItinerary/StayStep.js:284,287
<Paper className="stepLine" elevation={0} style={{ marginLeft: '0px' }}>
  <Box className="header">
    <Box className="type-icon">
      <Tipi />

------------------------------------------------------------

This is especially useful for finding unDRY code in large codebases that you are new to or aren’t wholly familiar with

I’ve been an IFTTT user for a while but only recently have I been trying to use it to its fullest extent. Apps like IFTTT & Zapier are an early version of augmenting ourselves.

Take a look at this example .tern-project file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

{
  "libs": [
    "browser",
    "ecma5",
    "ecma6"
  ],
  "loadEagerly": [
    "./node_modules/abacus-notepad-component/dist/*.js",
    "./node_modules/activity-component/lib/*.js",
    "./node_modules/component-popup/src/popup.jsx",
    "./server/**/*.js",
    "./server/*.js",
    "./client/src/js/**/*.js"
  ],
  "plugins": {
    "node": {}
  }
}

We’re using TernJS and it’s loadEagerly option to have intelligent & dynamic autocomplete available in our JavaScript editor setup. It works anywhere from vim to Visual Studio Code. But I digress.

As out project grows, we add more entries. As our number of projects grow, it will likely get copied all over the place, including onto other developers’ machines that you do not control. Even if you did automate (and even control) the propogation and maintenance of this file, you’d have a problem: the module names, file paths and file names aren’t normalized. A file path is an address. Addresses are normalized in society because they serve a purpose that is not reached if humans cannot make assumptions about them. When identifying structures within the United States of America, addresses are generally normalized to meet the following assumptions:

• The first piece of the identifier is real number, with the vast majority being integers (a small percentage have a vulgar fraction appended). Most importantly, the overwhelming majority of US streets have the odd numbers on one side of the street and the even numbers on the other. Anecdotal evidence shows this to be even more important than chromatic sequence when locating a structure
• The second piece is almost always the name of the thoroughfare touching the land nearest the official entrance to the structure
• The (optional) third piece is a sub-identifier representing that identity of the unit within the structure identified by the preceding and succeeding pieces
• The fourth piece is the city name
• The fifth piece is the state name
• The sixth piece is the Zone Improvement Code or ZIP code. It consist of five- and four-digit integer separated by a hypen (or “dash”). This is, in my opinion, one of the weaker parts of the address system as most folks do not know the 4 digit appendage that was introduced in 1983, nor do they usually know many ZIP codes other than their own

Just as the home address system in the USA does, a file path convention will have stronger and weaker aspects to it. All the same, we all know having an address system is better than none, so why would you not have one for files? In retrospect it becomes an obvious choice.

Take a look at an improved .tern-project file, taking into account the lessons of the address system

1
2
3
4
5
6
7
8
9
10
11
12
13
14

{
  "libs": [
    "browser",
    "ecma5",
    "ecma6"
  ],
  "loadEagerly": [
    "./node_modules/abacus-*-component/lib/**/*.js",
    "./lib/**/*.js"
  ],
  "plugins": {
    "node": {}
  }
}

What changes did we make?

• Name all our team’s custom components using the format teamname-modulename-component
• Always put our transpiled/consumable JavaScript files in a folder named lib, organized into appropriate subfolders& always using the extension .js. This is the convention, whether it’s a small package or an application

That’s it! We went from 6 entries to 2 just like that.

Happy filemaking!

A representative of an organization called TestDome Ltd contacted me a couple days ago. Today, they pinged me again, asking “Any thoughts on this?”. What they wanted my thoughts on was whether I could use their service of automating using programming quizzes to filter out folks in the (often long-winding) hiring process. My answer was as follows:

My thought is that this type of automation cannot solve the type of hiring problems we have at Netflix. Programming tests embody an extremely poor evaluation of senior programmers. Senior programmers leverage their past experience to effectively combine the best existing solutions in such a way that they’ve created some new, maintainable and sustainable. Senior programmers are good at working with others. They conform to, while incrementally improving, coding style and standards. The only way to pre-suppose about these things is looking through open source contributions, Stack Overflow answers, behavior on Twitter and then pair programming with them. An automated programming quiz merely tests how long they’ve spent doing automated programming quizzes.