πΏ - 2024 DAY 9 SOLUTIONS -πΏ
Day 9: Disk Fragmenter
Megathread guidelines
- Keep top level comments as only solutions, if you want to say something other than a solution put it in a new post. (replies to comments can be whatever)
- You can send code in code blocks by using three backticks, the code, and then three backticks or use something such as https://topaz.github.io/paste/ if you prefer sending it through a URL
FAQ
- What is this?: Here is a post with a large amount of details: https://programming.dev/post/6637268
- Where do I participate?: https://adventofcode.com/
- Is there a leaderboard for the community?: We have a programming.dev leaderboard with the info on how to join in this post: https://programming.dev/post/6631465
Haskell
This was fun, I optimized away quite a bit, as a result it now runs in 0.04s for both parts together on my 2016 laptop.
In part 1 I just run through the array with a start- and an end-index whilst summing up the checksum the entire time.
In part 2 I build up Binary Trees of Free Space which allow me to efficiently search for and insert free spaces when I start traversing the disk from the back. Marking the moved files as free is omitted because the checksum is calculated for every file that is moved or not moved directly.
::: spoiler Code
:::
Uiua
Just a port of my Dart solution from earlier really, and it shows, as it takes about 30 seconds on the live data.
(edit: I just noticed the little alien in the code
(β β β|β β|β)which is literally flipping the stack (β―Β°β‘Β°)β―οΈ΅ β»ββ»!)How to read this
Try it live!
(edit: improved. Part1 is instant, part2 is about 17sec, but the alien has left)
Haskell
Quite messy
Was really blanking on how to do this one nicely, so a bunch of stacked loops it is...
Also ended up writing two separate solutions for the first and second part, since I couldn't get acceptable performance otherwise. Still takes half a second on my machine, mainly on the second part.
This is technically the second implementation, the first one took minutes to calculate, so I wasn't really okay with stamping it as my solution-of-choice.
Can definitely still be improved, but I've been poking and prodding at this code for hours on end now, so it's long past time to let it sit for a while and see if I get any better ideas later.
::: spoiler C#
:::
Julia
Oh today was a struggle. First I did not get what exactly the task wanted me to do and then in part 2 I tried a few different ideas which all failed because I changed the disk while I was indexing into it. Finally now I reworked part 2 not moving the blocks at all, just using indexes and it works.
I feel that there is definitely something to learn here and that's what I like about AoC so far. This is my first AoC but I hope that I won't have to put this much thought into the rest, since I should definitely use my time differently.
::: spoiler Code
:::
PYTHON
Execution Time: Part1 = 0.02 seconds. Part2 = ~2.1 seconds. total = ~2.1 seconds
Aiming for simplicity over speed. This is pretty fast for not employing simple tricks like trees and all that.
:::spoiler code because of text limit and this code being slow, I put it in a topaz paste: [ link ] :::
Edit:
New version that is using a dictionary to keep track of the next empty slot that fits the current index.
Execution Time: Part1 = 0.02 seconds. Part2 = ~0.08 seconds. total = ~0.08 seconds 80 ms
:::spoiler code you can also find this code in the Topaz link: [ link ] :::
Edit: final revision. I just realized that the calculating for "last_consecutive_full_partition" was not necessary and very slow. if I know all the next available slots, and can end early once my current index dips below all next available slots then the last_consecutive_full_partition will never be reached. This drops the time now to less than ~0.1 seconds
Probably Final Edit: I found someone's O(n) code for OCaml. I tried to convert it to be faith fully in pure python. seems to work really really fast. 30-50 ms time for most inputs. seems to scale linearly too
:::spoiler FastCode
So cool, I was very hyped when I managed to squeeze out the last bit of performance, hope you are too. Especially surprised you managed it with python, even without the simple tricks like trees ;)
I wanted to try it myself, can confirm it runs in under 0.1s in performance mode on my laptop, I am amazed though I must admin I don't understand your newest revision. π
Just to let you know, I posted the fastest python version I could come up with. Which took heavy inspiration from [ link to github ]
supposedly O(n) linear time, and does seem to work really fast.
Thanks! your Haskell solution is extremely fast and I don't understand your solution, too. π lol
My latest revision just keeps a dict with lists of known empty slots with the length being the dict key, including partially filled slots. I iteratively find the slot that has the lowest index number and make sure the lists are properly ordered from lowest to highest index number.
looking at the challenge example/description, it shows a first pass only type of "fragmenting". we can be confident that if something did not fit, it can just stay in the same spot even if another slot frees up enough space for it to fit. so just checking if current index is lower than the lowest index number of any of the slot lengths would just be enough to stop early. That is why I got rid of
last_consecutive_full_partitionbecause it was slowing it down by up to 2 seconds.in example, even if
5555,6666, or8888can fit in the new spot created by moving44, they are staying put. Thus a first pass only sort from back to front.I only now found your edit after I had finished my previous comment. I think splitting into two lists may be good: one List of Files and one of Empty Blocks, I think this may not work with your checksumming so maybe not.
Thank you for the detailed explanation!, it made me realize that our solutions are very similar. Instead of keeping a
Dict[Int, List[Int]]where the value list is ordered I have aDict[Int, Tree[Int]]which allows for easy (and fast!) lookup due to the nature of trees. (Also lists in haskell are horrible to mutate)I also apply the your technique of only processing each file once, instead of calculating the checksum afterwards on the entire list of file blocks I calculate it all the time whenever I process a file. Using some maths I managed to reduce the sum to a constant expression.
yeah, I was a bit exhausted thinking in a high level abstract way. I do think that if I do the checksum at the same time I could shave off a few more milliseconds. though it is at like the limits of speed, especially for python with limited data types(no trees lol). Decently fast enough for me :)
edit: I also just tested it and splitting into two lists gave no decent speed up and made it slower. really iterating backwards is fast with that list slice. I can't think of another way to speed it up past it can do rn
Trees are a poor mans Sets and vice versa .-.
ah well, I tried switching to python's
set()but it was slow because of the fact it is unordered. I would need to use amin()to find the minimum index number, which was slow af. indexing might be fast butpop(0)on a list is also just as fast.(switching to deque had no speed up either) The list operations I am using are mostly O(1) timeIf I comment out this which does the adding:
so that it isolates the checksum part. it is still only 80-100ms. so the checksum part had no noticeable slowdown, even if I were to do the check sum at the same time I do the sorting it would not lower execution time.
Thank you for trying, oh well. Maybe we are simply at the limits.
so if I look at each part of my code. the first 4 lines will take 20 ms
The part1 for loop will take 10 ms.
The for loop to set up
next_empty_slot_by_lengthwill take another 10 ms.The part2 for loop will take 10 ms, too!
and adding up the part2 checksums will add another 10 ms.
So, in total, it will do it in ~60 ms, but python startup overhead seems to add 20-40 ms depending if you are on Linux(20 ms) or Windows(40 ms). both are Host, not virtual. Linux usually has faster startup time.
I am not sure where I would see a speed up. It seems that the startup overhead makes this just slower than the other top performing solutions which are also hitting a limit of 40-60 ms.
no way, someone is able to do it in O(n) time with OCaml. absolutely nutty. lol
Thank you for the link, this is crazy!
Python part 1
This is working for the demo, but not for the actual data. I'm a bit lost on why.
I'm always moving one (file)part at a time, so that should be fine... I think.
The fact that I need [:-2] suggests that I'm doing something wrong in parsing the input I guess...
C
First went with a doubly linked list approach, but it was quite verbose and we're dealing with short runs (max 9) anyway so back to the flat arrayβ. Plenty fast too - on my 2015 PC:
::: spoiler Code
:::
https://github.com/sjmulder/aoc/blob/master/2024/c/day09.c
Also did 2016 day 6 because reasons and I think it turned out real nice!
::: spoiler Code
:::
https://github.com/sjmulder/aoc/blob/master/2016/c/day06.c
Nim
Wrote ugly-ass code today, but it was surprisingly easy to debug and fast.
Solution:
Part 1: Parse data into a sequence of blocks and empty space like in example (I use
-1for empty space) and two indexes. First index goes 0 -> end, second index starts at the end. When we encounter empty space -> we use value from second index and decrement it (while skipping empty spaces). Repeat until both indexes meet at some point.Part 2: Parse data into sequence of block objects and try to insert each data block into each empty space block before it. Somehow it all just worked without too many bugs.
Runtime (final version): 123 ms
Codeberg repo
Rust
A bunch of fiddling with indices and sizes. In part 1 the disk is simulated in a Vec, for part 2 files and spaces are represented by their offset and size, collected in separate lists. Then these values are changed as necessary, with a whole bunch of
mut. In particular, files are never moved within the list of files, only their offset changes.::: spoiler Solution
:::
Also on github
Rust
Pretty poor performance on part 2, was initially 10s, got down to 2.5s, but still seems pretty poor.
Found a cheaty way to get sub 1s:
Might be possible to correctly do this in the find_slot code, so that if it fails to find a slot, it never searches for that size again.
edit:
500ms, I can go to sleep now.
haha, kept going at it, got it down to 4ms, by tracking where the searches ended, and starting from there next time.
Definitely done now :D
Haskell
Unoptimized as hell, also brute-force approach (laptops are beasts).
::: spoiler Spoiler
:::
Dart
I just mapped the filesystem onto a list holding value at each block (with -1 for spaces), and manipulated that.
It's slow, but it's honest work.
::: spoiler Slow version
:::
Updated version
Massive speedup from implementing a modified KnuthβMorrisβPratt algorithm -> around 0.5sec runtime for part 2.
I really don't understand why efficiently matching a sublist isn't a builtin function. Implementing it manually was half an hour of unneeded head-scratching.
Nushell
As I'm still on holiday and normal languages are a PITA to type on a phone, I though I'd try a compiled scripting language. I'm not very familiar with it so it took longer to code and also it's been running the first reduce step for 35 minutes so I've missed the 24h cutoff π
You coded this on a phone, with a touchscreen keyboard? Not sure who is more impressive, you or the unicode wizard :D
I just ran this on a laptop - it worked (part one only) but took 4h28m21s so Nushell is not a language for AoC (or I just coded it very poorly).
I think once your runtime hits 4 hours, the minutes and seconds stop being relevant :D
Is it 4hrs of 100% CPU?
One core was busier, but it wasn't at 100%. My Rust code yesterday was the same, perhaps it's taking too much time accessing memory.
The time was wall time (as per Starship's output) but it was still waaay too slow to bother with again!
I'll buy "better than default on screen keyboard", but I doubt it'll be better than a physical keyboard. Would be nice if there was a virtual keyboard that let you easily modify the layout. I keep hitting
.instead of space on my phone.Haha, thanks but we both know they're #1 by a country mile. I think my phone's now downclocking as it's burning up and still hasn't spat out an answer after two hours, so I technically haven't completed it yet!
Edit: Calling it for now, I might figure out later why it's so slow (there's some easy but minor gains to be made but I'm guessing there's some O(awful) going on that the input's blown up)
Haskell
Not a lot of time to come up with a pretty solution today; sorry.
::: spoiler Ugly first solution
:::
Second attempt! I like this one much better.
Edit: down to 0.040 secs now!
It will always be a wonder to me how you manage to do so much in so few lines, even your naive solution only takes a few seconds to run. π€―
Aww, thank you <3
It's just practice, I guess? (The maths degree probably doesn't hurt either)
Maths degree at least explains the choice of language
C#
python
::: spoiler solution
:::
Kotlin
No lists were harmed in the making of this code.
::: spoiler Solution
:::