Spyke

Syndicated from the fediverse. Read and engage on the original instance.

View original on piefed.zeromedia.vip

I rewrote "tldr" in C, it's 20 times faster.

Seriously! According to the time command in fish, cldr (my program) takes around 4-5 ms to print files, tldr takes 80-100ms. (PS, on my server, cldr actually takes less than 1 ms to fetch the file, while tldr takes closer to 150 ms. That's 150 times faster!!!)

This started a couple days ago when a friend of mine alerted me to the fact that the official tldr repo now takes ai contributions for their pages (aka the cheat sheet files that are printed) and for the actual program code. She then showed me a repo called ethical-tldr which does not allow for the use of ai contributions in the pages or program code.

However, their even though their client doesn't allow vibe coding, it is still written in Python, which itself takes ai contributions.

So, I decided to learn C from scratch and make my own tldr client that's faster and better than theirs in *every way! Introducing cldr! I am especially proud of the list and search functionality because the interface is just gorgeous.

Note, ABSOLUTELY NO AI was used in the making of this, I learned and wrote everything on my own in just the past 3 days, I don't think I even copied much or any code from websites.

*There's still like two things I haven't implemented that the official client does, but they're very minor and I'd be willing to bet that 99% of you won't know the difference.

I rewrote "tldr" in C, it's 20 times faster.https://forgejo.zeromedia.vip/girlnamedzero/cldrOpen linkView original on piefed.zeromedia.vip
185

65 replies

lemmy.world

What your project desperately needs is a description of what it does. I had to follow the links to ethical-tldr back to the original tldr to see what it's supposed to do.

76

It's basically short-form manpages, so it gives you a one-liner to describe what a CLI utility does, and then just lists a few concrete invocations to show off different use-cases for it.

One other big difference to manpages, though, is that the actual texts aren't packaged by your distro, but rather downloaded from a server.

13

This is an instance of that recurring trope of coding something better out of pure spite

52
piefed.social

So, I decided to learn C from scratch
...
on my own in just the past 3 days

Really impressive, great job! What resources did you use for learning?

29

Mostly just w3. Occasionally I had to look up some stuff for more “niche” use cases, but for the most part I just learned from w3.

I already have experience with programming in general, so the hardest part was learning the syntax, and strings… oh god strings are so bad in C, I had to make a new function called strfmt that works like printf but just for returning a formatted string.

22

yes! However, I found it very frustrating to use and it kept leading to ludicrous errors beyond my wildest dreams. So I made my own function that works better and easier.

3

Glad you are enjoying C. As one of my proffesor's espoused, "It is God's language". As a C++ developer, I cannot recommend {fmt} enough.

4
sinwellreply
infosec.pub

Those are method names in C, so I’m afraid you’ll have to take it up with a guy named Dennis who isn’t very talkative these days.

5
feddit.nl

Not sure what you mean. You said that you had to make a new function and decided to call it "strfrm". The standard is actually the opposite afaik. You should be more verbose and not use contractions. Something like "string_format" is far easier to read.

4
sinwellreply
infosec.pub

Ah I didn’t see OP’s strfmt you’re right I’m wrong

2
feddit.nl

And I didn't notice that you weren't OP. I guess we're even? Haha

2

I was just following the naming convention as other string methods, like strlen, strcmp, strcat, (string length, string compare, and string concatenate, respectively), mine is string format, so I did strfmt!

3
lemmy.world

So... C code is faster than python. We knew that. Thanks for the better performing rewrite. Now do it in rust 🤓

26
Creatreply
discuss.tchncs.de

In all distros I've recently used this is what gets installed when you install the "tldr" package with the package manager. And for good reason.

20

I'll admit I have no idea what makes one of the Rust implementations preferable over the other. Nor can I go looking at the code myself now that GitHub has hosed itself again

5
Valmondreply
lemmy.dbzer0.com

Why in rust? I mean it would be less faster, right? If speed is of the essence ofc.

1
communismreply
lemmy.ml

Performance of compiled Rust and compiled C are comparable. Of course, compiling Rust is slower though.

4

Yeah, but it's slower because it has compile-time safeguards and guardrails that eliminate some families of really dangerous memory and IPC errors.

3

Because "rewrite it in rust" is its own meme and needs no particular reasoning.

4

The performance metrics are cool (though unscientific) but speed is definitely NOT of the essence lmfao

TLDR is not a high throughput real time application. You run it like once per use, probably at a rate of less than once per day. Then you spend a minute reading the output. We're talking about minutes-per-year time savings here.

3

Actually not. Rust can be even faster than C, although it's generally similar in performance to C. The point is that it's more secure because of the way it handles memory.

2

We use CLDR quite a bit — and coincidentally, we often jokingly call it TLDR.

2
esa
discuss.tchncs.de

You might want to have a glance at the OpenSSF compiler hardening guide, and maybe try out ASAN (-fsanitize=address).

E.g. I get

[nobody@bb14e99a9c8e cldr]$ ./build/cldr --update --language C.
caution: filename not matched:  ethical-tldr/pages.C./*/*.md

=================================================================
==454==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x7fa48a32c0c1 in malloc (/usr/lib/libasan.so.8+0x12c0c1) (BuildId: b8a4241051a1621937fdc46e867ba7ecb56d96ea)
    #1 0x7fa489e8be0f  (/usr/lib/libc.so.6+0x8be0f) (BuildId: 503200d7fda94a5dc6058d7e0694e5d1dcb2e372)
    #2 0x7fa48a2f2761 in vasprintf (/usr/lib/libasan.so.8+0xf2761) (BuildId: b8a4241051a1621937fdc46e867ba7ecb56d96ea)
    #3 0x561d84213d8c in strfmt (/home/nobody/cldr/build/cldr+0x6d8c) (BuildId: 2a0f13efc2c9b12a67e7dd810a2cbdfd1866e53d)

SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
18

Ah damn! That's one of the things I had told myself I needed to make sure I checked, but I forgot! Thanks! I'll look into this tomorrow. Like I said, I literally learned C over the last 3 days to make this, so I'm always happy to learn more!

17

Update to this, ~~I actually just fixed that bug, and ~~I went through the first section of the hardening guide. Changes are now pushed to main and I released v1.2.1

16
programming.dev

This looks like a colossally bad idea:

  if (!strcmp(platform, "windows") && !isSpecificPlatform) {
    debugLog("Falling back to rm -rf", 2);
    debugLog(strfmt("rm -rf exited with code %d",system(strfmt("rm -rf %s", path))), 3);
  }

In addition,

However, their even though their client doesn’t allow vibe coding, it is still written in Python, which itself takes ai contributions.

Per the Makefile, this project is built using GCC. GCC itself accepts AI contributions, provided that the code is "legally insignificant" (meaning less than around 15 lines of code) or is part of test cases: https://gcc.gnu.org/ai-policy.html

18

For the first part: You're probably right! I'm not sure of a safer way to do it though. This specific part is an artifact of the original method I used, which I don't think would have worked on windows (according to some stack overflow post), but the current code might actually work fine, I just don't have access to anything running windows and would rather not set up a VM.

Second: That's highly disappointing! I think their policy is fairly well thought out, in a way that allows for accountability and auditability, so I might be okay with continuing to use it, I'll have to think about it. If you know of an alternative I could use, I will happily take suggestions!

2

To add to the last comment, I'd also suggest not mixing logging and business code. That is to say that you should put the system call on its own line, so that it is clear that it's not just a logging call:

  if (!strcmp(platform, "windows") && !isSpecificPlatform) {
    debugLog("Falling back to rm -rf", 2);
    int result = system(strfmt("rm -rf %s", path)); 
    debugLog(strfmt("rm -rf exited with code %d", result), 3);
  }

It may just be a me problem, but I have a tendency to skim over the logging lines unless I'm intentionally reading code carefully, in which case the system call may get overlooked

6
programming.dev

You can call rm safer manner, but you'll need to fork and use one of the execl functions directly. That way you don't have to worry about escaping the path. The old implement would potentially delete unrelated data if the path contained whitespace, and could even execute arbitrary commands, since it essentially runs bash -c "rm -rf $path"

3

Those are really good points! I've been sanitizing other inputs for shell breakage, but never considered checking this. Either way, I have since removed the rm -rf part of the code, instead opting to only use my "good" method of traversing the directory

1

Update to my comment: I changed it up a little bit to remove that bit of code. You can optionally uncomment part of the code to re-enable it, in case the current implementation doesn't work on windows

1

My reasoning is simple: I had never heard of it.

But also, this is an incredibly simple program and a really good way to help me learn C. Additionally, it seems like tealdeer accepts ai generated submissions, though generally discourages it. I aim to make my project as free of ai as I possibly can.

10
lemmy.world

I wish people still had shame.

Python isn't for command line utilities that you intend to distribute to others. It's for personal one-off scripts.

I mean, FFS, if you're gonna vibecode something, why not pick a fast language? The fact that they can't even tell the LLM to use the right technology shows that they are a complete imbecile

6

Python isn’t for command line utilities that you intend to distribute to others

wat

12
darklamerreply
feddit.org

Python isn't for command line utilities that you intend to distribute to others.

Sorry, but you're wrong about this. Python can be an excellent choice for many command line utilities intended for distribution to others.

3

I don't enjoy making my computer work 20 times harder than it has to

1
programming.dev

Dependancies

[Optional] wget can be installed for less reliance on other projects using ai, if wget is not found, curl will be used instead

  1. Small typo: dependencies.
  2. Is curl using AI in its code now?

Thanks for your work!

5

1: thanks! I’ll fix that soon.

2: this is according to the same friend I talked about, but it would be good to double check her knowledge

2

excellent. however if you are making a point about AI not being as good s a quality developer, you are missing the point.

CEOs don't give a shit about quality and effectiveness. it is all about getting rid of labor costs. once all he few remaining providers of service are absorbed, we take what they give and pay through the nose. monthly.

4

C ftw, but seriously, calling an entire language(python) AI slop is going too far, though I will admit python has been responsible for slop code even before LLMs. My opinion on the matter is, efficient use of LLMs will usher a push to go back to hardware level languages like C, as all the benefits of python(except the awesome collection of libraries it has unfortunately accumulated) are kind of moot when you have a machine capable of outputting any boilerplate you ask of it; beyond that I do not trust LLMs to write logic unless its a well known and named algorithm like reversing an array or a linked list.

0