Spyke
rust·Rust Programmingbynothingness

How to run background jobs?

Is there any library for the queueing mechanism?

What's used by the most - Cron? But a task or rather script executed by Cron won't access to the context of an application. Meaning, a task will have be an independent unit. Whereas I want is a library to use inside a project such that it'll have access to everything.

Anything similar to Sidekiq exist in Rust?

View original on lemmy.world
7
Comments17
SuddenlyBlowGreenreply
lemmy.world
let mut scheduler = Scheduler::with_tz(chrono::Utc);

scheduler.every(10.minutes()).plus(30.seconds()).run(|| println!("Periodic task"));

scheduler.every(1.day()).at("3:20 pm").run(|| println!("Daily task")); 

scheduler.every(Tuesday).at("14:20:17").and_every(Thursday).at("15:00").run(|| println!("Biweekly task"));

Damn, that a really ingenious and intuitive use of the builder pattern.

Kudos to the devs!

7
SuddenlyBlowGreenreply
lemmy.world

You could schedule a task at 1 PM, then generate a random number between 0-60 inside that task, wait that many minutes, then launch the actual task.

2
kevincox
lemmy.ml

For my project I just run them. I'm using async so it is just spawning a task in the background, but you can do the same with threads. Don't underestimate the number of threads that you can run on a modern computer.

If you want some sort of throttling you can stuff tasks into a queue and just run N background threads pulling them off an processing them.

If you need durability then you start to have more trouble. This is where I would start looking at a library. IDK if there are any libraries that handle logging, retries and similar, but if not you can probably get the basics down pretty easily.

1
nothingnessreply
lemmy.world

For my project I just run them.

How would you "just run" a task every 30 minutes? Every 5 hours? Once a day?

0
kevincoxreply
lemmy.ml
std::thread::spawn(|| {
    loop {
        std::thread::sleep(std::time::Duration::from_secs(30*60));
        do_job();
    }
});

Works pretty well. Maybe add a bit of code to crash the whole process on panic or some other logging. Wastes a few KiB of memory per loop but probably not a major issue. Doing this with async will waste only the tiniest amount of memory.

2
kevincoxreply
lemmy.ml

It depends. Sometimes you can just put an exit call at the end of main to kill the thread. If you want to attempt graceful shutdown then usually I just use a boolean shutdown flag. Then the loop becomes while !shutdown.get() {

1
kevincoxreply
lemmy.ml

Best option is probably to add a wrapper around the thread that re-spawns it. But you can also just catch panics in the loop.

1
nothingnessreply
lemmy.world

How would you re-run it multiple times then? An internal should be progressively greater. How would you terminate it if it continues to produce an exception?

0
ikornaselur
lemm.ee

I looked into this last week for recurring tasks in an actix-web project and ended up picking fang, but haven't really battle tested it.. But seems to do what I need. I'm using cron tasks with Postgres as the backend to run a recurring task every minute.

1
vaalla
discuss.tchncs.de

Can you use async in your project?

If Yes, you can spawn a task that will listen on a channel. If you need to run them in parallel probably you can find a mpmc channel.

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

Don't know any crate just for this.

1
nothingnessreply
lemmy.world

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

How would you do it every 30 minutes? Every 5 hours? Once a day?

0

You reached the end

How to run background jobs? | Spyke