For a stretch, Ballpark Genius’s home run leaderboard had Jonah Heim on top. One home run. Congrats to him, I guess, but even he’d tell you that’s not how any of this works. The query behind it filtered WHERE team_id = 0 to get season-total rows instead of per-team splits, correct for a player who’s been traded, MLB gives those a team_id = 0 aggregate row, and wrong for literally everyone else, who only has per-team rows. That left about seven hitters and eight pitchers as the entire eligible pool for every leaderboard on the site. Whoever had the highest single stat among that tiny, mostly-irrelevant group won.
Fixed with one query instead of two: DISTINCT ON (player_id) ORDER BY player_id, team_id ASC picks the team_id = 0 row for anyone who has one and falls through to their single per-team row otherwise, then the outer query sorts and takes the top N like it always should have. Checked it live after: Judge, Alvarez, Schwarber, et al, back where they belong.
Small, dumb, expensive bug. The kind where the fix is three lines and the entire distance between “leaderboard” and “not a leaderboard” was one wrong WHERE clause the whole time.
Ballpark Genius has a champion/challenger loop that tunes a new prediction model every night and promotes it if it’s actually better, in theory, for the folks reading who haven’t seen the earlier post about that loop. It promoted heuristic_v9 over the baseline weeks ago. The live API kept using the baseline’s math anyway, just with a v9 label stapled to it.
generateGamePrediction() had the weights hardcoded as literals, eraSwingMax, parkRunFactorWeight, homeAdvantageBase, the works, regardless of which model version the database said was champion. The tuner could promote a model with meaningfully different weights and the prediction endpoint would go on doing exactly what it had always done. Self-improving in name only (a title I’ll admit describes more of my code than I’d like).
Fix was to actually read the champion: getCurrentChampion() loads its weights once at the top of generateGamePrediction, and calculateWinProbability takes them as a parameter instead of reaching for a constant. Checked it against a real game, Nationals at Marlins on May 10th: 38% home win probability under v9, 46% under v0, an 8-point swing from wiring alone, no new data, no new model, just finally using the one that had already won. Across that week’s slate of 92 games, v9 picks the home team 47% of the time vs v0’s 65%. The old hardcoded weights had a structural home-field bias that was never supposed to survive a real promotion.
Somewhere on this project there’s a lesson about verifying the thing you shipped is the thing that’s running. I already knew that lesson. Here we are anyway.
I took 30m to experiment with Node v20 yesterday, initially focused on experimental permissions. I noticed features that have been introduced since Node 14 that apply to Sports Card Investor’s engineering org, e.g. corepack.
Node/CorePack
One of those is Node’s corepack, which holds pnpm as an add’l official package manager that it will manage for you. You enable corepack like so. You declare which package manager and version thereof your Node project is using like so
Node::Test
Another is the introduction of the Node-native test package, which works really nicely with assert. assert has always been useful for Node projects but the introduction of test makes it so that jest is just not necessary for many projects
Experimental Permissions
Node 20 now has a method for locking down which system aspects your code can access & have side effects on. I’m predicting that this feature will also be useful for debugging issues related to third-party code interacting with the file system (e.g. “what the hell is mutating this file?”). Here is an example of making FS permissions read-only and limited to the current directory & its descendents. Experimental Node API usage triggers warning messages by default, I’ve also used an environment variable to silence those—the point of that repo is using experimental APIs
Tying it all together
By running these tests with pnpm I’m combining corepack, test & --experimental-permissions to verify that FS reads are enabled in the project directory, while FS writes are not
cluster.on('exit', worker => console.log(`worker ${worker.process.pid} died`)); } else { // Workers can share any TCP connection // In this case it is an HTTP server createServer((req, res) => { res.writeHead(200); res.end('hello world\n'); }).listen(8000);
console.log(`Worker ${process.pid} started`); }
Changes from the current Node.js ‘cluster’ docs include:
Using import over require
Using named imports over the default where feasible
Using clearer, more semantic identifier names, e.g. currentCPU v. i
Replacing archaic terminology, e.g. Master w/ Main
> $ node index.mjs ⬡ 14.15.1 [±main ●] Main 57290 is running Worker 57292 started Worker 57295 started Worker 57291 started Worker 57294 started Worker 57293 started Worker 57296 started Worker 57298 started Worker 57297 started Worker 57299 started Worker 57300 started Worker 57302 started Worker 57301 started
One of the constants in my software dev career has been a predilection for reducing redundant code as much as feasible. There’s a JSX-fluent CLI tool that been helping me do that at iTRVL.
Running
npx jsinspect ./packages/agent/src/components/Itinerary/EditItinerary/*Step.js will give you a set outputs like so:
I’ve been an IFTTT user for a while but only recently have I been trying to use it to its fullest extent. Apps like IFTTT & Zapier are an early version of augmenting ourselves.
We’re using TernJS and it’s loadEagerly option to have intelligent & dynamic autocomplete available in our JavaScript editor setup. It works anywhere from vim to Visual Studio Code. But I digress.
As out project grows, we add more entries. As our number of projects grow, it will likely get copied all over the place, including onto other developers’ machines that you do not control. Even if you did automate (and even control) the propogation and maintenance of this file, you’d have a problem: the module names, file paths and file names aren’t normalized. A file path is an address. Addresses are normalized in society because they serve a purpose that is not reached if humans cannot make assumptions about them. When identifying structures within the United States of America, addresses are generally normalized to meet the following assumptions:
The first piece of the identifier is real number, with the vast majority being integers (a small percentage have a vulgar fraction appended). Most importantly, the overwhelming majority of US streets have the odd numbers on one side of the street and the even numbers on the other. Anecdotal evidence shows this to be even more important than chromatic sequence when locating a structure
The second piece is almost always the name of the thoroughfare touching the land nearest the official entrance to the structure
The (optional) third piece is a sub-identifier representing that identity of the unit within the structure identified by the preceding and succeeding pieces
The fourth piece is the city name
The fifth piece is the state name
The sixth piece is the Zone Improvement Code or ZIP code. It consist of five- and four-digit integer separated by a hypen (or “dash”). This is, in my opinion, one of the weaker parts of the address system as most folks do not know the 4 digit appendage that was introduced in 1983, nor do they usually know many ZIP codes other than their own
Just as the home address system in the USA does, a file path convention will have stronger and weaker aspects to it. All the same, we all know having an address system is better than none, so why would you not have one for files? In retrospect it becomes an obvious choice.
Take a look at an improved .tern-project file, taking into account the lessons of the address system
Name all our team’s custom components using the format teamname-modulename-component
Always put our transpiled/consumable JavaScript files in a folder named lib, organized into appropriate subfolders& always using the extension .js. This is the convention, whether it’s a small package or an application
That’s it! We went from 6 entries to 2 just like that.
A representative of an organization called TestDome Ltd contacted me a couple days ago. Today, they pinged me again, asking “Any thoughts on this?”. What they wanted my thoughts on was whether I could use their service of automating using programming quizzes to filter out folks in the (often long-winding) hiring process. My answer was as follows:
My thought is that this type of automation cannot solve the type of hiring problems we have at Netflix. Programming tests embody an extremely poor evaluation of senior programmers. Senior programmers leverage their past experience to effectively combine the best existing solutions in such a way that they’ve created some new, maintainable and sustainable. Senior programmers are good at working with others. They conform to, while incrementally improving, coding style and standards. The only way to pre-suppose about these things is looking through open source contributions, Stack Overflow answers, behavior on Twitter and then pair programming with them. An automated programming quiz merely tests how long they’ve spent doing automated programming quizzes.
It’s clear I love dynamic languages. I like metaprogramming. I like DRY code. I hate repeating myself (at least when I write code—I repeat myself a lot in person). Objective-C and JavaScript are both dynamic languages and those are the two langs I’ve written produced the most open source code with.
I also like reliable and fast languages. That’s why, although they’re dynamic, I don’t favor Ruby or Python. Python is the better of the two, but I still can’t justify creating the type of web services I write in Python.
That was all setup for the following: Just as Swift is less dynamic than Objective-C, new JavaScript is less dynamic than old JavaScript. Examples:
import|export syntax vs. CommonJS
More types and implementations of type systems
Static analysis in the language itself isn’t the only reason we’ve gotten less dynamic. Build time tools such as Browserify, static analysis via ESLint, type checking via Flow and several other tools have given us greater safety at the expense of the former wild west freedom.
While I strongly dislike giving up dynamism, I have a much stronger dislike for unDRY (WET?) code. There are a couple recent additions to JavaScript that require a little more thought up front but result in DRYer, safer (and after some re-training of your team) more expressive code. I’m talking about object shorthand syntax and computed property names. Technically computed property names have a duality between safety and danger, but that’s why I love JS.
Object shorthand syntax reduces errors and reader’s overhead by taken advantage of that fact keys and the variables assigned to them should be the same anyway. I’ve always done it that way and viewed variance from that as evidence of not having thoroughly evaluated the why and how of semantics in your application. This syntax is especially valuable in React components, where passing props is common:
No joke, I’ve seen plenty of code where the variables were just as arbitrarily named differently from the keys.
Computed property names allow one to (dangerous but powerful skill) dynamically create method names on an object while declaring the object or (safe skill) use constants to name your methods while declaring the object. We’ve always been able to do the following:
As an ADHD-addled obsessive who’s been writing JavaScript since 1998, the drive to stay avante garde is nothing new to me. Whether it’s collecting the latest comic books, tech tomes or guitar effects pedals—I’ve always yearned to get my hands on the latest. From childhood on I’ve used a proven set of techniques and principles to guide the way I consume fresh information. The same techniques that helped me transition from a construction worker/dishwasher to a Netflix software architect are the same techniques I use today to upgrade from Babel 5 to Babel 6.
Now on to JavaScript Fatigue specifically. JSF has relatively recently entered into the web developer’s lexicon. However it’s been a part of technical work for much longer. The difficulty of keeping up-to-date in our industry has been there for many years, but only recently has it become a social pressure. An in-depth exploration of the reasons for the rise of “JavaScript knowledge as fashion” are not in the scope of this article, but the drivers include:
Github’s socialization of code
Twitter being a key forum in which we examine our place in the industry
A rapid increase in salaries—and the subsequent gold rush of smaller-better-faster code jocks. This includes the me-first virus in our industry (see this)
Regardless of the causes, the following 6 principles are the cure for JavaScript fatigue that works for me:
Automate, automate, automate. Any sites that you repeatedly visit for information should be automated as feeds via services like IFTTT.com. An example of doing this can be seen here.
Eliminate all information inputs that are not essential to being the best programmer/manager/artist/human you can be. I follow something resembling an inverted Mad Max version of Pareto Principle here. If a Google Group, newsletter or Twitter user do not produce life-enhancing content to you at least 80% of the time, eliminate it. That Ruby on Rails user group that was really active in 2007 but it a shell of its former self? Unsubscribe. That high school buddy Bradley that was fun in 2003 but now posts 100% negative rants? Bye Bradley.
Constantly replace your previous realms with new ones. If you’ve already covered following 100s of mobile development brothers on Twitter, try following some of the web development sisters. Learn about new areas of life from as many different types of people as possible! Default to saying yes, then revisit and say no aggressively. Say yes to trying new meetups, modules & software, but don’t stay too long if they’re not working for you after you’ve given it the ol’ college try.
Ensure you’re constantly around experts in different but related fields. Contrary to popular wisdom, only associating with teammates that are focused on your realm can result in suboptimal performance as you will spend more time debating choices than making them and executing on them. On my team at Netflix, I’ve an author & former Digital Humanities Specialist from Stanford, a design-savvy D3 expert & a Data Scientist with a business degree around me at all times. We’re all multi-disciplinary and hold one another accountable, but defer to one another on the implementation details of our respective areas of expertise. I learn about new things—so do they—and we focus more energy on learning than arguing.
Allow the wisdom of the crowd to lead you to treasure, but don’t let mob mentality dictate which gems you put in your rucksac to take back to camp. Crowd-sourced wisdom is o’plenty on the web and can be found at Product Hunt, by following key individuals on Twitter & by subscribing to the popularity feeds of Github.
Never study when you’re fatigued unless you’re in-the-zone. Ensure your body and mind are primed to effectively take on new information. There’s no point in going through your info feeds when you’re too knackered to move any of the info into long-term memory. Physical health is deeply tied to mental health too—take walks in the sunshine frequently.
