I created a TUI for systemd/systemctl called isd (interactive systemd).

It provides a fuzzy search for units, auto-refreshing previews, smart sudo handling, and a fully customizable, keyboard-focused interface for power users and newcomers alike.

It is a more powerful (but heavier) version of sysz, which was the inspiration for the project.

This should be a huge timesaver for anybody who frequently interacts with or edits systemd units/services. And if not, please let me know why! :)

ww520 2 years ago

Looks nice.

One thing I found systemd really confusing was its treatment of ExecStop in a service script. ExecStart is the command to run when systemd starts the service at system boot up (or when a user tells systemd to start the service). However, ExecStop is run when the starting command has finished running. You have to set RemainAfterExit=yes to have the desired function of running the stop command on system shutdown or on user stopping the service. ExecStop is basically the "on-cleanup" event rather than "to-shutdown-the-service" event.

Cyph0n 2 years ago

I think about them as “on start” and “on stop”.

It is important to keep in mind that systemd is tailored towards daemons. So if your service just runs a command that eventually exits, you need to explicitly tell systemd to treat it differently than a daemon.

Edit: As others noted, you’re probably looking for oneshot + RemainAfterExit.

rcxdude 2 years ago

It is a little asymmetric, because 'ExecStart' is actually normally 'Executable that is the service', not just script that starts the service, but I think that's a hangover from the self-daemonizing approach to init scripts.

Cyph0n 2 years ago

True, but it still makes sense to reason about them this way. Say you have an HTTP server:

- on start: start the server

- on stop: do nothing, because you are already terminating the server

But suppose you need to perform an additional task when the server is terminated. That is where you would add a ExecStop command or script.

ww520 2 years ago

Then ExecStop is basically on-cleanup, not to-stop. ExecStart really is to-start, not on-start. In the httpd server case, ExecStart runs the httpd command.

Cyph0n 2 years ago

If that helps you understand it better, then sure.

All I am trying to say is that the name of the option makes sense as-is: it literally runs the provided command(s) on service stop (“execute on stop”).

Similarly, ExecStart literally means command(s) to execute on start.

If the command runs a blocking daemon/server (usually the case), then the server will be implicitly stopped by definition - because if you’re stopping the systemd service, you’re interrupting any commands that are still running/blocking.

SuperNinKenDo 2 years ago

I think part of the problem is with how one "naturally" or "habitually" thinks of a Service. From systemd's perspective/terminology, the Service is the thing that starts and stops. But whether because it is inherently more intuitive, or because of how daemons traditionally worked on *nix, the mind tends to think of the process which the Service starts as the thing which starts and stops. I'm not able to double check currently, but I also think that systemd isn't totally consistent with the mental model it bases the choice of keywords on when it comes to ExecStop, because with that mental model one would expect ExecStop to only run when the Service is stopped from systemd, but I'm fairly sure it runs in other cases when the process started with ExecStart exits, but the service isn't necessarily stopped. I could be wrong about that though.

yread 2 years ago

ExecStop works the way you want for type=forking

ww520 2 years ago

Still has the same problem even with type=forking. Only way to get it working was RemainAfterExit=yes

shawnz 2 years ago

I think you actually probably want type=oneshot (and also RemainAfterExit=yes) for the kind of service you're describing

brirec 2 years ago

This was ultimately what I needed to do when I wrote a systemd service that managed some firewall rules. It really was a footgun though, what with having essentially different meanings/purposes for ExecStop whether you’re doing a Type=forking, a Type=oneshot, or a Type=oneshot with RemainAfterExit=yes.

And relatedly, I honestly have no idea when I’d want to use ExecStartPre, or multiple ExecStarts, or ExecStartPost, and so on.

ww520 2 years ago

Having different semantics with different proprieties on the same command is really confusing.

shawnz 2 years ago

I would argue the semantics of ExecStop are always the same. It's the command that's executed to stop the service. On the other hand, what it means for a service to be "running" or "stopping" naturally depends on what type of service it is (i.e., is it a daemon or not?)

ww520 2 years ago

> the command that's executed to stop the service

That’s what is assumed. But in reality it runs after the started process stops.

shawnz 2 years ago

Yes, so whether the service is stopping as a result of the process exiting, or whether you requested the service to stop manually, it will run the ExecStop in either case.

That makes sense to me personally. What would be the more intuitive design in your mind?

ww520 2 years ago

Stopping as a result of the process exiting or requested the service to stop are two very different things. Systemd overloads the term ExecStop for different semantics, relying on different property settings. That's where the confusion comes from.

drougge 2 years ago

The name sounds like it means "this is how I want you to cause the service to stop" to me (and clearly to others as well). That would be symmetrical with ExecStart meaning "this is how I want you to cause the service to start". If it runs after the service stopped it should be called "ExecAfterStop" or something like that.

shawnz 2 years ago

That is what ExecStop means. It specifies how you want to cause the service to stop. But the lifetime of the service isn't exactly the same thing as the lifetime of the process that got started in ExecStart.

Maybe think about it this way: ExecStart is what the system will run to transition the service from the "starting" state to the "started" state. ExecStop is what the system will run to transition the service from the "stopping" state to the "stopped" state.

For a service with RemainAfterExit=no (the default), you enter the stopping state right away once the processes that got started in ExecStart exit. That's useful when you are starting some long lived process as a service, and in that case there is usually no need for an ExecStop. But semantically, ExecStop has the same meaning either way -- it's what needs to be run, if anything, to transition the service from the stopping state to the stopped state.

drougge 2 years ago

I have now found the documentation for ExecStop (in systemd.service(5)), which hopefully improves my understanding.

It definitely seems to be both "cause to stop" and "after (unexpected) stop" in one. You can look at $MAINPID to see which case you have. This design apparently makes sense to you, but to me and several others in this thread a service that has already stopped isn't in need of being stopped and shouldn't execute commands intended for that. (There is a separate ExecStopPost for "after stopping, for any reason".)

yencabulator 2 years ago

Think of it as an enum, Type branches the logic.

    enum Service {
        Exec { ExecStart: String, ... }
        Forking { ExecStart: String, ... }
        OneShot { ExecStart: String, ... }
    }
You can argue that sometimes that ExecStart could be a different term, but it'd still end up being the same across multiple enum variants.

ww520 2 years ago

Yes. ExecStart works the same for all the cases. ExecStop works differently though. While ExecStart is the event to kick off the command for the service, ExecStop is not. The asymmetric semantics are where the confusion comes from.

BenjiWiebe 2 years ago

It's been enlightening to me to read through some of the distro-provided .service files to see what can be done, with services I'm more of less familiar with.

wasted_intel 2 years ago

Love this. I use raw CLI commands until it hurts, and have recently embraced tools like lazygit/lazydocker to get visibility into otherwise opaque system/tree states, and it’s been a huge level-up.

I have several user and system level services I manage, but debugging them is tedious. Your opening line that lists common commands and their pain points really resonated with me.

I’m on NixOS, so editing immutable unit files directly won’t work, but the service discovery, visibility, and management will be really helpful. Nice work!

kai-tub 2 years ago

I am also a NixOS user and this is exactly what motivated me to work on this project!

I am planning on adding some "guides" in the documentation but in short: You should check out `systemctl edit --runtime` for debugging units on NixOS. It makes debugging sooo much easier.

ripley12 2 years ago

This looks very good, thanks for sharing! I maintain a similar project and working with the systemd/dbus APIs has been pretty painful; eager to try this and see what I can learn from it.

https://github.com/rgwood/systemctl-tui

gurgeous 2 years ago

This is incredible! I will use this a ton. Only thing missing is a deb package...

Fnoord 2 years ago

At long last, systemd-client: merry meet! Next step is such a TUI for non-Linux such as macOS, FreeBSD, Windows. For macOS I use LaunchControl.app but it isn't a TUI.

Just one thing: I had to do

  $ uv tool install git+https://github.com/isd-project/isd
instead of

  $ uv tool install https://github.com/isd-project/isd/isd@latest
and uvx wouldn't work at all, version: uv 0.5.21. That said, uv is way more quick than pip(x) so I just switched.

kai-tub 2 years ago

Yeah, silly me. It is fixed now, thanks for letting me know!

mahoro 2 years ago

So cool that with uv it becomes so easy to install such tools.

What's missing in the install routine is uv installing this tool ignoring the Python dependency. My box has 3.10 and isd won't work with it. Fixed with `-p 3.13` option. May be worth mention in the docs.

kai-tub 2 years ago

Hey, thanks for mentioning this! I have included the flag in the docs.

johnchristopher 2 years ago

> If you ever became frustrated while typing:

Hey, that's me ! (And I love systemd !)

I haven't installed it yet so quick question: can it connect to remote host ? I often use systemctl --host <hostname> status foo.service (status, timers, logs etc. )

kai-tub 2 years ago

Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`. But there is also an appimage that should make it (hopefully) easy to run it on remote servers.

Either way, feel free to open an issue and I will have a look at it.

diggan 2 years ago

> Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`.

Sounds like you could easily support it by letting users pass in $REMOTE_HOST and when you use your `systemctl` wrapper, add `$CMD --host=$REMOTE_HOST`, after that everything should work as before.

kai-tub 2 years ago

I will investigate it!

https://github.com/isd-project/isd/issues/6

throeurir 2 years ago

I can not install this stuff on remote servers and docker images. I would like multiple backbends to execute commands and gather informations (local, ssh, docker).

It should be installable locally, and run commands on remote machine via ssh! And via 'docker exec' commands.

jchw 2 years ago

> If you have nix installed, this AppImage will not work! Either use the nix or uv installation instructions!

Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.

Interestingly though, on Nix v2.24.11, I can't use the provided Nix command either:

    $ nix run https://github.com/isd-project/isd
    error:
           … while fetching the input 'https://github.com/isd-project/isd'
    
           error: Failed to open archive (Unrecognized archive format)
Even if that did work (you could adjust it into a Git URL to make it work) it would probably not be ideal since Nix has a native GitHub fetcher that is more efficient. I think this should be the actual Nix command:

    nix run github:isd-project/isd
Anyway, this is cool. I actually wanted to make a similar thing using systemd's DBus API and Qt instead of a TUI and even started writing code for it, and if you wanted to I'm sure you'd find that the DBus API probably provides all of the functionality you would need (admittedly it is a lot easier to just call `systemd -H` than to implement your own SSH tunneling, though.) It kind of frustrates me that systemd and modern Linux in general is absolutely teeming with data and interfaces that could be exposed and make administering systems, especially desktop systems that were traditionally very inscrutable, much easier. e.g. in the past, how did you know what was going on when an xdg autostart app failed? Now with systemd running xdg autostart apps in some desktops, it would be really easy to provide a GUI that can show you the failed autostarts and even provide a GUI log viewer, and yet somehow, such a tool does not seem to exist, at least in the realm of things that are maintained and relatively feature-complete. Rather frustrating.

jchw 2 years ago

I am completely confused as to why this comment seems to be poorly received. Can someone explain?

speed_spread 2 years ago

That may be a bit of nix backlash, don't sweat it.

I agree Linux could use better system APIs than "put file here" and "run these commands" which are much more error prone than making calls to properly documented interfaces.

kai-tub 2 years ago

> Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.

Yeah, it get's complicated and I don't want to recommend it and explain the details. In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues. For example, you could install your EDITOR via home-manager on Ubuntu. isd would start correctly because systemctl is "accessible" but if you open your EDITOR under `/nix` it wouldn't find it, which is super confusing as a user. -> So it is just easier to say to use the nix installation method if you are already using it :D

And sorry for the wrong docs, it is fixed now.

I also agree with your frustration. Personally, I would really enjoy working on such a tool but it wouldn't be an easy task, and who would support the work? This TUI had a manageable scope but it was still quite a bit of work. So I don't see myself investing too much into "higher-level interfaces".

PS: I have no idea why your post is ill-received :/

jchw 2 years ago

> In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues.

That makes a lot of sense. There's not really much trivial that I think you can do to deal with that.

> PS: I have no idea why your post is ill-received :/

It's not really a big deal, I was just surprised and wondered if anyone who was compelled to vote down might come back to explain why.

elric 2 years ago

Looks great, well done. It's a shame that it's needed at all. The vast majority of my interactions with systemd are trivial: (re)starting a service, looking at a log file to figure out what's wrong, and making sure a service starts on boot. I find it baffling that the ergonomics of systemd for those common tasks are so lacking. But the TUI seems to help, so thanks.

And sure, systemd it's more deterministic and includes the kitchen sink, unlike initd.

Thankfully these days I can automate most of such interactions out of existence, so I no longer feel the burning hatred that I once did. More like a smoldering ember.

kai-tub 2 years ago

I mean, I get why they don't want to "bloat" systemd with a complex/opinionated TUI, but I would've also really liked a more "upstreamed" interface. Though, I guess making systemd more beginner-friendly/accessible is not really that important for their funding?

gchamonlive 2 years ago

How's security handled? Not in terms of system permissions which Linux handles well, but in terms of guarantees that it can't be hijacked and remotely controlled by an external attacker.

kai-tub 2 years ago

Author here: I also find this an important thing to ask yourself when you are running applications/scripts that do anything with sudo and which is why I have written a fairly in-depth "Security" section on the isd documentation page:

https://isd-project.github.io/isd/security/

Let me know if anything is missing!

gchamonlive 2 years ago

As a suggestion, since your repo is open I think you can leverage sonarcloud without costs. It would make for another independent check that your code does what's intended and for instance won't ping a remote control server, either via dependency attacks or via malicious pull requests that could dodge reviews.

Where I work we also use defectdojo to catalogue and manage CVRs in our projects, but it's more involved to setup the testing pipeline and deploy the required services.

crabbone 2 years ago

On search and editing system unit files:

1. My life improved a lot after I found that you can do "systemctl status $PID" and systemd will find what service (if any) is responsible for the process in question. This has been a life saver many, many times. But, more search would still be welcome, especially for cases when the system fails to boot, or fails to reach a particular target etc.

2. I think systemd didn't go far enough with unit files. The motivation was to escape the hell of Shell scripts, where each system was defined in its own unique way, and was failing in a dozen of unique ways. While, initially, it might have seemed that a simple INI-style format could manage to describe service requirements... I think, it's way overdue to realize that it doesn't. And sysadmins on the ground "fix" that by embedding more Shell into these configuration files, bringing us back to the many unique ways a service will fail. Perhaps, having a way to edit these unit files so that it doesn't expose the actual format may lead to improvement in the format (more structure, more types, templates).

hedora 2 years ago

This looks almost as easy to use as slackware’s init/syslog back in the Linux 0.9x days.

If you add a sane cli with tab completion support, it’ll come full circle.

jonwest 2 years ago

Maybe a bit of a gross simplification, but would you say this is analogous to something like k9s for Kubernetes? It looks handy, to say the least.

wint3rmute 2 years ago

That was my first thought as well, seems to fill a very similar niche, just for systemd instead of kubernetes

kai-tub 2 years ago

I have never used or touched k9s nor Kubernetes, so... Maybe?

renewiltord 2 years ago

Haha this is great. It’s funny that others also end up doing this enable, start, journalctl, start dance. Good stuff. I shall try it.

My only problem is that I wish there were a way to install it on my machine and have it connect to a remote systemctl but that probably is a lot of work to reliably work (port may not be open etc etc).

kai-tub 2 years ago

Maybe it is possible with `systemctl --host`... Though, I would have to make quite a few changes to the UI design since it would mean that there are "multiple modes". One for local and multiple remotes. So... Check back in a month or so :D

noisy_boy 2 years ago

Maybe you can keep it uniform by always presuming "host" as one of the parameters (and localhost just happens to be the default for it).

willm 2 years ago

Nice! How did you find working with the Textual library?

kai-tub 2 years ago

It was great! The extensive documentation and the vast amount of widgets were definitely the main reasons why I have chosen to do it in Python/Textual rather than other languages/libraries.

Though I don't agree with all design decisions, and there are some bugs, having such well-maintained documentation and clean repo makes it easy to dive into the project and understand what is going on and how other components are built.

Thanks for the library!

yonatan8070 2 years ago

Looks super cool, I've been working quite a bit with systemd recently, and typing systemctl and journalctl + their flags gets old rather fast.

Can it connect to remote hosts like you can with systemctl --host?

airstrike 2 years ago

This looks super well done and polished. The rich docs are crazy detailed for something that was just released. You're really setting the bar for what other projects should do. Congrats!

kai-tub 2 years ago

Thanks!

I've put quite a lot of work into it and wanted the project to give a good first impression. This was important to me since I believe these "niche" TUI libraries need to immediately show what problems they are solving or how they save time, as users need a reason to _use_ an additional abstraction layer.

airstrike 2 years ago

Absolutely! Really well done. I now think you're the best person to do the same for ffmpeg lol

egberts1 2 years ago

Keep in mind, systemd is still a default-allow access control list (ACL) and has a very long road ahead in form of Linux security, as long as CAP_SYS_ADMIN continuea to exist.

May make it easier to customize but it doesn't close the security loopholea like SELinux, GRSecurity, TOMOYA, or AppArmor does.

https://www.cyberciti.biz/tips/selinux-vs-apparmor-vs-grsecu...

detuur 2 years ago

Yes yes and a reminder that BSD Jails are better than anything Linux does and a bunch of other dead horses we like beating on the regular around here.

Which is a fanciful way of saying that I don't understand the relevance of your comment at all to the topic at hand, which is an interactive frontend.

egberts1 2 years ago

You read across as a security-indifferent.

surajrmal 2 years ago

Honestly it's not all or nothing. It really depends on your threat model. If you assume all native code is both not malicious and incapable of being compromised to run untrusted code then the existing model works. At that point it's more important to focus your security efforts on how native code ends up running, and therefore the distro specific packaging ecosystem.

Now for many that threat model is not sufficient as they both run increasingly less trustworthy software, obtained by less trustworthy mechanisms such as npm or off a website, or simply want to protect against bugs that cause otherwise non malicious software from being compromised and resulting in security incidents. I'm in this latter camp but we can't ignore the fact that there are many who happily operate in the former. There also exist solutions such as we browsers with their sandboxes and VMs that somewhat fill the requirements for running untrusted software for these individuals.

alduin32 2 years ago

This is amazing, I had a bunch of work to do on some remote hosts this night and tried using isd as much as possible, to see how it can help me.

Some feedback :

- it is.. relatively slow ? especially when focusing on different panes (tab/shift+tab). on my machine it takes at least half a second to react

- the unit list is missing page-up/page-down handling

- in some unit attributes, the ordering of some values frequently changes (for example, on unstarted services, in `TriggeredBy`)

- it could be interesting to integrate the output of `systemd-analyse security`

Nice work !

kai-tub 2 years ago

- That the tab pane switching is the slow part is a bit weird. Is there a difference when you see all units and a subset of units? Maybe open an issue and I can provide more possible settings that could be changed.

- Jep, page up/down handling will be added. Thanks!

- Not quite sure when this happens. Maybe an example screen shot (before/after) could help?

- Yeah, that is definitely on the roadmap.

alduin32 2 years ago

I've opened an issue about the reordered attribute values (+ another bug I found). Tab pane switching stopped being slow (I suppose my laptop was simply overloaded, and I'm too used to things being fast even when it is).

larusso 2 years ago

Need to see this in action. The example videos overuse the fuzzy search for commands in my opinion. Makes it hard to concentrate what the actual action/information is when a huge part of the screen changes. Not to say I don’t like fuzzy searchable command prompts. But it can be clearer to either just type out a command or have a menu system. As I said I need to check this out in action for myself.

cdiamand 2 years ago

The security section is good to see. Thanks for that!

kai-tub 2 years ago

Funny that you mention that. I honestly thought not too many people would care about it.

Though I am by far no security specialist. Please let me know where I can improve the section!

Saris 2 years ago

Isn't `service nginx status` a shorter way to do it? I've always done that instead of the longer systemctl commands.

rglover 2 years ago

Thank you for shipping this. Need to test it but going to add this to the deps bundle for my deployment service (relies heavily on systemd). Looks like it would make day to day maintenance/debugging far less of a headache.

marginalia_nu 2 years ago

Looks very useful. Standard tooling for systemd is such an annoying maze to navigate.

udev4096 2 years ago

This would be extremely helpful. Although, I would love a feature where you can add different linux hosts and manage them all from a single place, using ssh

eliaspro 2 years ago

Most systemd functionality has built-in support for targeting remote hosts and local machines (containers, VMs) and mostly utilizes SSH under the hood to do so.

Check the -H and respective -M flags of many systemd CLI utilities.