• 0 Posts
  • 44 Comments
Joined 1 year ago
cake
Cake day: June 10th, 2023

help-circle






  • Could use Polars, afaik it supports streaming from CSVs too, and frankly the syntax is so much nicer than pandas coming from spark land.

    Do you need to persist? What are you doing with them? A really common pattern for analytics is landing those in something like Parquet, Delta, less frequently seen Avro or ORC and then working right off that. If they don’t change, it’s an option. 100 gigs of CSVs will take some time to write to a database depending on resources, tools, db flavour, tbf writing into a compressed format takes time too, but saves you managing databases (unless you want to, just presenting some alternates)

    Could look at a document db, again, will take time to ingest and index, but definitely another tool, I’ve touched elastic and stood up mongo before, but Solr is around and built on top of lucene which I knew elastic was but apparently so is mongo.

    Edit: searchable? I’d look into a document db, it’s quite literally what they’re meant for, all of those I mentioned are used for enterprise search.



  • My dad was big into Sierra an Apogee so of what I can recall, be one of the incredible machine, king’s quest 6, duke nukem 2 or command keen. Also remember playing math and word rescue at like… 5, shareware was more than enough to keep me occupied at that age. First console game though was altered beast on the genesis.











  • I was trying to get on the list at mt work when I got a hardware refresh this year, I dislike large laptops and the dev spec is a 17" thinkpad (which imo has the left CTL and fn keys backwards, breaks muscle memory when changing between computers) but I’m docked most times but when I’m not the battery is terrible, maybe a handful of hours. Probably due to corporate crapware, but at least the arm macbooks stand a chance, my partner has an m1 mbp and she doesn’t bother charging it most workdays or work with it plugged in, she doesn’t need to. We were playing factorio the other night and she was moonlighting into her desktop, she got through a day’s work, a bunch of hours of game streaming and some of the next work day, that should be the expectation for a normal device.

    Apple in my view really understood mobile devices, they had the hands down best trackpad for a long time, a fantastic keyboard, great display, a form factor you can actually carry around and as far as I recall, even the intel macs had better battery life.