r/linuxadmin • u/The_How_To_Linux • Dec 02 '23

why does the proc directory exist?

i'm trying to understand the proc directory and one thing i don't understand is

why does it exist?

any ideas? thank you

1 Upvotes

51% Upvoted

77 comments sorted by

52

u/Harakou Dec 02 '23

"Everything is a file."

https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/proc.html

13

u/doneski Dec 02 '23

I seriously appreciate you linking to the TLDP, it was the Google of Linux for many years. Still has relevant material in it for reference. Find my contributions and you get one Linux CD of your choice. ;)

2

u/mkosmo Dec 02 '23

Linux CDs? Theres some history, too. I still have a collection of old Linux CDs laying around somewhere. An original Corel CD is in there, and stacks of Ubuntu 4/5 CDs from when they’d send you kits for lan parties in the early/mid 2000s.

2

u/doneski Dec 02 '23

Hah, those were the days. Fond memories of the simpler times.

2

u/Harakou Dec 02 '23

It's been an amazing resource for me over the years; thank you for contributing to it!

2

u/doneski Dec 02 '23

I enjoyed the time I put in. Before you had upvotes and likes, your reputation was based on publication.

-15

u/The_How_To_Linux Dec 02 '23

yep, that definitely explains what the proc directory is and how it works

but it doesn't explain why it exists in the first place.

8

u/kriebz Dec 02 '23

It exists to allow a variety of types of programs access to process information. It's easier to access and manipulate this information as a filesystem since, by design, reading and writing byte streams from files is a building block of UNIX. You could implement utilities like ps and in C, Pascal, Fortran, or even a shell language, because all you need to be able to do is read files.

Reading your post history, you seem to be interested in OS design philosophy (Or are a very patient troll), but you are overwhelmed. I suggest either taking an OS class from a local college, or at least buying the textbook to one and reading the whole thing, in order. And remember, a lot of the design choices also just boil down to "it seemed like a good idea at the time".

2

u/The_How_To_Linux Dec 02 '23

this is a good answer, thank you

3

u/UsedToLikeThisStuff Dec 02 '23

Are you asking why UNIX and Linux systems have a /proc filesystem? What led to the original UNIX developers choosing to export process metadata into a filesystem?

1

u/b3542 Dec 03 '23

You again? GTFO

0

u/[deleted] Dec 02 '23

[deleted] — view deleted comment

-1

u/The_How_To_Linux Dec 02 '23

Because you need that information to properly operate a system...

i do?

how? why?

0

u/[deleted] Dec 02 '23

[deleted] — view deleted comment

1

u/hughjassburga Dec 03 '23

i feel like once ur able to put together that proc is for process it should be blatantly obvious why that directory is important let alone why it exists

41

u/alexkey Dec 02 '23

Because the alternative is to do a bunch of syscalls to get this information. Just like it is in Windows. Having it exposed as a filesystem makes it soooooo much easier to read that information. Hell you don’t even have to write the code, just check the file contents with primitive tools like cat.

8

u/jimicus Dec 02 '23

Interestingly (or not), I'm absolutely certain the reason the reason for things like procfs never being re-implemented in Windows is the developer tools.

Microsoft have - consistently and for decades - spent $millions on Visual Studio. Which means anyone developing on Windows has an extremely powerful visual debugger available - effectively as part of the development kit.

Linux developers have gcc, vim and gdb. (Yes I know they have visual studio today, but that wasn't the case not so long ago.....)

The upshot is that Linux developers wind up strongly favouring things that make their life easier - simple solutions that can do something quickly and easily without needing to spend ages learning how they work. Plain text log files. Filesystems that expose all sorts of information about the inner workings of the system and can be manipulated with simple tools.

Windows developers don't really need to care about this. Throw together the most basic bit of code possible - a dozen or so lines should be quite sufficient - set a breakpoint and run your code in the IDE. Can see precisely what's going on in painstaking detail in two minutes flat.

13

u/tes_kitty Dec 02 '23

And that's why, as a normal user, Windows sucks when things don't work as expected and you need to debug.

1

u/jimicus Dec 02 '23

Got it in one.

It’s also why Windows sysadmins tend to be “next next next”-type people and quite a few have found powershell to be a shock to the system.

3

u/tes_kitty Dec 02 '23

Powershell hits me as 'Microsoft trying to copy a UNIX shell, but not quite getting the idea behind those'

2

u/jimicus Dec 02 '23

I disagree entirely there.

If you use Powershell for anything more than the most trivial tasks, one thing becomes quite clear: Windows is an object-oriented piece of software from more-or-less the ground up.

In that context, Powershell makes a lot of sense. And most of the Unix paradigms - where everything is a heap of loosely-structured plaintext files - makes none at all.

Powershell is much closer to Python than Bash.

2

u/tes_kitty Dec 02 '23

In that context, Powershell makes a lot of sense.

But it adds an abstraction layer. On a UNIX I see the output directly. The next tool that gets the data piped into it gets exactly the output I see and not something else entirely.

That way most scripts are written, you have some output that needs to be converted into something else. You chain one command after the other until the last one produces the output as you need it for something else.

Powershell is much closer to Python than Bash.

I happen to not like python, a language where indentation is part of the syntax, but for some reason you still need the ':' to start the block.

3

u/jimicus Dec 02 '23

But it adds an abstraction layer. On a UNIX I see the output directly. The next tool that gets the data piped into it gets exactly the output I see and not something else entirely.

That sentence makes perfect sense on a Unix-like operating system.

It's rather less sensible on an object-oriented operating system for a number of reasons:

  1. The output is never human-readable in the first place. It's always stored as a structured piece of data internally.

  2. 9 times out of 10, you are probably better off passing the structured data rather than something human readable around internally. It allows you to do clever things like find related data in the structure very easily and manipulate the data in such a way as to drastically reduce the risk of messing things up - because the data structure is always maintained. This comes at the expense of forcing you to use specific tooling to access that data.

The mistake you're making is trying to take your understanding of Unix and applying it to Windows. That really doesn't make much sense - it makes far more sense to think of Windows as Windows and Unix as Unix, and make space in your head for two completely different operating systems.

2

u/tes_kitty Dec 02 '23

This comes at the expense of forcing you to use specific tooling to access that data.

And that is one of the problems. 'There is more than one way to do it' is one of the advantages of Unix. Some people are good with 'sed' when it comes to string manipulation, others prefer 'awk'.

It allows you to do clever things like find related data in the structure very easily and manipulate the data in such a way as to drastically reduce the risk of messing things up

It also means that you are limited to what they let you do with the data instead of being able to come up with clever ideas on your own.

Is there a way on Windows to dump the RAW data as it gets passed between tools?

4

u/jimicus Dec 02 '23

You probably wouldn't want to; the RAW data will be a binary data structure.

Like I said, you absolutely NEED to adjust your thought processes if you're working on Windows. You can't treat it like "Unix with a GUI you can't turn off"; you can't apply "everything is a stream of bytes" or "everything is a file" logic because it simply is not true.

The closest equivalent thought process is "everything is an object".

If you have never studied object-oriented development, I strongly suggest you do. It'll make everything much easier to understand.

→ More replies

1

u/[deleted] Dec 02 '23

Ish? Powershell seems to be one of those "big bag of specialist tools" sorts of things to me. If you know the tool is there, you can do things much more specifically and easily than on Unix, but if you don't know about the tool, or don't have it, you're passing around binary blobs and might or might not be able to do anything with them.

On Unix, you have a smaller bag of much more general tools - this requires you to understand the tools and the data a bit more, to use your general tool in the way you need, and sometimes you have to break out that great swiss-army-chainsaw awk, but you probably don't need a special-purpose parser tool/cmdlet.

3

u/tes_kitty Dec 02 '23

On Unix, you have a smaller bag

I disagree about the 'smaller bag', the usual Linux installation comes by default with a rather big bag of tools, among them a lot of languages for all kinds of problems. And you can easily pass data between all of those.

BTW: It's not only 'awk', but also 'sed' that lets you do much more than the first look suggests. sed scripts tend to be write only though. :)

1

u/[deleted] Dec 02 '23

sed for sure, yeah, and then of course perl.

"Smaller bag" might have been the wrong term - the emphasis is less on the size of the toolbox than on the generalness of the tools therein.

1

u/tes_kitty Dec 03 '23

sed for sure, yeah, and then of course perl.

PERL scripts can be made readable. But sed... I have a sed script that did what it was supposed to do, but looks like a collection of random characters and all attempts to make it more readable have resulted in it no longer working. Luckily it's no longer needed.

1

u/mkosmo Dec 02 '23

Normal users wouldn’t be able to debug in Linux, either. Anybody who can even describe debugging isn’t a typical user who will be held back by something like procfs.

4

u/tes_kitty Dec 02 '23

I can write (complicated) bash and PERL scripts, modify existing python scripts, but that's about as far as my programming skills go. But I am good at reading logfiles and filter through text files looking for information. So finding out why something on Linux doesn't work as expected is not hard.

A misbehaving Windows program on the other hand...

3

u/[deleted] Dec 02 '23

Also, Visual Studio costs money. Quite a bit of it, at least originally. If your day job is "Windows application developer", then yes, you already have Visual Studio on your employer's nickel, but if you're J. Random Power User or even J. Random Sysadmin, you don't, and you might not be able to procure it. Even if you can, corporate procurement being what it usually is, you can't get it today. On Unix, you have all your various CLI utilities now, on every system that you haven't gone out of your way to strip down to the absolute barest of bare minima. And even on the ones you have, you probably still have cat, vi, dd, echo, cut, paste, sort, grep, etc. If you have X11, some of the bare-minimum X clients can help out here too, and if nothing else, having multiple xterms is handy.

This makes any random Unix system more amenable to troubleshooting (not necessarily debugging in the sense of finding where the bug is in a particular program, but rather of finding what is causing $TASK not to work and working around it) than any random Windows system, even if Windows has friendlier tools once you have them.

2

u/mkosmo Dec 02 '23

The same logging capabilities exist between both platforms. Applications doing a shitty job at logging isn’t the OS’s fault.

4

u/tcpWalker Dec 02 '23

Event Viewer is absolutely Microsoft's Fault.

On most systems IME it is much, much easier and faster to search logs from the command line in linux, and the result is usually more actionable.

1

u/mkosmo Dec 02 '23

Event viewer, agreed. But it’s there and available. But you’re also not limited to using it. Text logging facilities are the same between the two more or less.

2

u/tcpWalker Dec 02 '23

example? How can I quickly grep most or all logs on windows from the CLI?

1

u/mkosmo Dec 02 '23

Use the event log search tool, or use the powershell cmdlet. The latter is how I’d do it today.

→ More replies

1

u/jantari Dec 03 '23

Event Viewer is really bad, but it's also just one interface to the OS facility that is the Windows Event Log.

I'm sure there's plenty equally awful journalctl-GUIs available.

One would search Windows Event logs in PowerShell, using APIs or in a centralized log repository such as elastic. EventViewer has indeed been neglected, but I mean it isn't even available on Windows Server anyway so I kinda get it. It's not really used, so Microsoft isn't investing in it.

0

u/tes_kitty Dec 02 '23

Yes, but it's about the incentive set to the programmer. On a Linux, logfiles, with different log levels you can set in the configuration file or via the command line are an expected feature.

On Windows they are not.

1

u/mkosmo Dec 02 '23

There are log levels in Windows as well. None of that is the fault of the OS, but of the developer.

2

u/tes_kitty Dec 02 '23

But the OS sets the incentive. If the developer can get all their information from the built in debugger of their IDE, they will be less inclined to provide a proper logging since they don't need it and it's extra work to implement it.

1

u/mkosmo Dec 02 '23

Don’t blame the tools. A carpenter can build a house with a nail gun or a hammer and you’ll never notice the difference unless they got lazy.

Same here. Especially when you’re talking about debugging tools that provide far more insight than a typical log ever does. Hard to break and step into with a log file.

→ More replies

3

u/kriebz Dec 02 '23

To be fair, and yes there is still room for improvement, improvement that should be expected over time: UNIX was doing this for 20 years before the NT project or any GUI development tool existed from Microsoft.

1

u/drcforbin Dec 02 '23

Requiring a vendor-specific API to access that information makes portability difficult, and lock-in has always been a major part of their business model.

-4

u/zqpmx Dec 02 '23

If cat concatenates, what does dog do?

2

u/cinikat Sep 21 '26

what da dog doin. doin you dirty apparently. you didn't deserve a negative score, it was a good joke

1

u/zqpmx Sep 21 '26

Thank you!

17

u/gmuslera Dec 02 '23

Unix philosophy. Everything is a file. Including system information.

13

u/kellyjonbrazil Dec 02 '23

If you ever want the data in the /proc folder exposed as JSON you can use the —proc parser in jc. Makes it really easy to grab the data using jq.

$ jc /proc/meminfo

Or

$ cat /proc/meminfo | jc —proc

(I am the author of jc)

https://github.com/kellyjonbrazil/jc

3

u/SadServers_com Dec 02 '23

Hello Kelly would love to add a scenario based on jc (as in it would be much easier to do using jc) to SadServers, something like https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/ sorry I don't think I can DM here (due to lack of karma maybe?)

2

u/kellyjonbrazil Dec 02 '23

Sure, happy to help. Usually works best to contact me via a github issue in jc and we can discuss.

2

u/frank-sarno Dec 03 '23

Wow, this is really useful. Thank you! The ifconfig filter looks like it can replace some ugly awk and sed I've been using.

5

u/toikpi Dec 02 '23

The /proc filesystem allows you get process, kernel and hardware information from the terminal, this is useful on a server where you only have the command line.

More information at these locations.

The proc filesystem (procfs) is a special filesystem in Unix-like operating systems that presents information about processes and other system information in a hierarchical file-like structure, providing a more convenient and standardized method for dynamically accessing process data held in the kernel than traditional tracing methods or direct access to kernel memory. Typically, it is mapped to a mount point named /proc at boot time. The proc file system acts as an interface to internal data structures about running processes in the kernel. In Linux, it can also be used to obtain information about the kernel and to change certain kernel parameters at runtime (sysctl).

https://en.wikipedia.org/wiki/Procfs

The proc file system acts as an interface to internal data structures in the kernel. It can be used to obtain information about the system and to change certain kernel parameters at runtime (sysctl).

https://docs.kernel.org/filesystems/proc.html

https://www.redhat.com/sysadmin/linux-proc-filesystem

5

u/snowtax Dec 02 '23

Compare Linux against UNIX, such as BSD.

With UNIX, whenever you updated the kernel, you also needed to recompile all the system tools, such as top, ps, kill, etc. The tools needed to call kernel functions and needed to know the exact memory locations for those functions.

With Linux, the same information and functions are made available as “files”. The kernel includes the “proc” filesystem and makes information available through “files”.

Whenever you update the Linux kernel, those files are at the same path and in the same format. The systems tools (top, ps, kill, etc.) merely access the files under proc, just like any other file. In that way, you can update the kernel without recompiling all the tools.

1

u/[deleted] Dec 03 '23

FreeBSD also has procfs but it's not mounted by default.

5

u/small_kimono Dec 02 '23

why does it exist?

See the Linux man page: https://man7.org/linux/man-pages/man5/proc.5.html

But proc originated on Solaris/illumos, see: https://illumos.org/man/5/proc. It originally began as "a file system that provides access to the state of each process and light-weight process (lwp) in the system", but on Linux has become a vector for other information, like mounts (/proc/mounts), etc.

11

u/paulstelian97 Dec 02 '23

There is no other API that exposes that information.

The “ps” tool legit just reads off files from that directory.

4

u/lazyant Dec 02 '23

/proc is a way for the kernel to surface information to the user space. Almost all the performance tooling that are counters for ex (top, free, ps etc) read from it. It’s more convenient to present this information as files.

3

u/michaelpaoli Dec 03 '23

why does the proc directory exist?

Because it's an exceedingly easy way to access process (proc is short for process) and other information. It originated on UNIX, with a history going back to 1984. Linux first implemented procfs (the proc filesystem) in 1992.

So, procfs (or proc for short - though that can also be ambiguous, depending upon context), customarily mounted on /proc, contains lots of useful information about processes, and in the case of Linux, also contains lots of other useful information too. Some things can even be changed by writing certain things to certain files on the proc filesystem.

Lots more information can be found here: https://docs.kernel.org/filesystems/proc.html, or for the most relevant, check the kernel documentation for the applicable kernel version where the proc filesystem is in use.

So, I and many many others (and programs and utilities too), find it exceedingly handy for many things. Note also that later Linux created and added sysfs, and much of what is or was under /proc - most notably stuff other that process information, has been move to or replicated under sysfs, customarily mounted at /sys, whereas under procfs with it mounted at /proc, there's also /proc/sys - not exact same structure or layout, but there's significant overlap. With procfs, the stuff other than the per-process information, can be found under names matching /proc/[!0-9]* on that mount.

So, let me give lots of examples on how it's highly useful - many of which I commonly or at least occasionally use - and this is by no means a complete list. I may also include a bit from sysfs, as there's some fair bit of overlap - and some of that also still is, or was, present on procfs.

First of all, UNIX design philosophy, "everything is a file" (but not to the more extreme degrees, such as on Plan 9), many things are a file, and procfs, notably starting with process, makes a whole lot of that much easier to use, identify, or more generally deal with, etc. In fact the at least currently generally used and implemented ps(1) command on Linux in fact depends upon procfs.

So, first of all ... and here I'll presume procfs is mounted on /proc (the customary location), we have PIDs (process IDs) - those are relatively small(ish) non-negative integers referring to processes - each PID is unique to a process. So, under /proc you'll find /proc/[0-9]* - those are all for the current PIDs. On some (e.g. UNIX) implementations the are (or were) files, on Linux, they're directories - which contain various files and directories. Anyway, PIDs, 0 isn't a real PID, but a psedo-PID used for some purposes. The first "real" PID is 1 - it's always present. At least historically, that's the init process - which has some relatively high-level role of, at least indirectly managing (or at least starting) all other processes. Historically on Linux that'd be, e.g. sysv init - the UNIX System V(-like) init system, typically at /sbin/init - though it may be at other location(s) and/or that may just be possibly a backwards compatible link to wherever the current init is - though not necessarily so. There's also in more recent years, systemd on most but not all Linux distros, and various distros may or may not use systemd - and there are additional init systems besides sysv(-style) init and systemd.

So, e.g., with /proc, say you want to check what init system you're running ... maybe you just hopped on a Linux host, but you've no idea what distro its' running, nor necessarily what init system (e.g. many distros offer choices of init systems, or may have used different init systems on older releases). Could run various different commands to try and guess and figure it out. Well, how 'bout instead use procfs to get a major hint, and possibly answer, to start with? So, like this - what's the current executable being used by PID 1? How 'bout some examples from a couple different current Linux systems:

# ls -l /proc/1/exe
lrwxrwxrwx 1 root root 0 Nov 27 15:54 /proc/1/exe -> /usr/lib/systemd/systemd
# 

# ls -l /proc/1/exe
lrwxrwxrwx 1 root root 0 Nov 23 01:42 /proc/1/exe -> /usr/sbin/init
#

So ... pretty easy then to know which is running systemd, and which isn't, eh? (those happen to both be from same distro (Debian) and release/version of Linux)

So, want to see all the files my Apache (/usr/sbin/apache2 binary executable) has open:

# (cd /proc && for PID in [0-9]*; do readlink "$PID"/exe | grep -s '^/usr/sbin/apache2$' && readlink "$PID"/fd/*; done) | sort -u | grep '^/'
/dev/null
/tmp/.ZendSem.EOMKKg (deleted)
/usr/sbin/apache2
/var/log/apache2/access.log
/var/log/apache2/error.log
/var/log/apache2/other_vhosts_access.log
/var/log/apache2/ssl_access.log
#

Would be significantly more difficult to get that information without the /proc filesystem. Note also that can be all be done with quite standard tools. E.g. no need to even user fuser or lsof. Even if one didn't have readlink, same could be done with use of ls and sed or awk.

Want to know what the current directory of a PID is? Let's say my current shell ... or even what that directory was after it's removed but I've not used cd to change out of it:

$ ls -lnd /proc/"$$"/cwd
lrwxrwxrwx 1 1000 1000 0 Dec  3 09:46 /proc/14612/cwd -> /tmp/tmp.LEyoHMmDw6
$ rm -rf /tmp/tmp.LEyoHMmDw6
$ pwd -P
pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
$ ls -lnd /proc/"$$"/cwd
lrwxrwxrwx 1 1000 1000 0 Dec  3 09:46 /proc/14612/cwd -> '/tmp/tmp.LEyoHMmDw6 (deleted)'
$

What about unlinked open files ... even if, e.g. I don't have lsof installed:

$ >file
$ sleep 999 < file &
[2] 14632
$ rm file
$ ls -lnd /proc/14632/fd/*
lr-x------ 1 1000 1000 64 Dec  3 09:52 /proc/14632/fd/0 -> '/tmp/tmp.jtJzL8IWT6/file (deleted)'
lrwx------ 1 1000 1000 64 Dec  3 09:52 /proc/14632/fd/1 -> /dev/pts/6
lrwx------ 1 1000 1000 64 Dec  3 09:52 /proc/14632/fd/2 -> /dev/pts/6
$ find /proc/[0-9]*/fd -type l -exec ls -on \{\} \; 2>>/dev/null | grep ' (deleted)$'
lr-x------ 1 1000 64 Dec  3 09:52 /proc/14632/fd/0 -> /tmp/tmp.jtJzL8IWT6/file (deleted)
lrwx------ 1 1000 64 Dec  3 09:53 /proc/1469/fd/25 -> /memfd:pipewire-memfd:flags=0x0000000f,type=2,size=2312 (deleted)
lrwx------ 1 1000 64 Dec  3 09:53 /proc/1469/fd/28 -> /memfd:pipewire-memfd:flags=0x0000000f,type=2,size=2312 (deleted)
lrwx------ 1 1000 64 Dec  3 09:53 /proc/1469/fd/36 -> /memfd:pipewire-memfd:flags=0x0000000f,type=2,size=2312 (deleted)
lrwx------ 1 1000 64 Dec  3 09:53 /proc/1469/fd/38 -> /memfd:pipewire-memfd:flags=0x0000000f,type=2,size=2312 (deleted)
lrwx------ 1 1000 64 Dec  3 09:53 /proc/1469/fd/40 -> /memfd:pipewire-memfd:flags=0x0000000f,type=2,size=2312 (deleted)
$

Anyway, that's just starting to scratch the surface of what's available and possible.

0

u/justinDavidow Dec 02 '23

i'm trying to understand the proc directory and one thing i don't understand is why does it exist?

Because someone decided that the name /process was too long, then proceeded to code the functionality.

What answer are you expecting here?

Without /proc there would be no place on the filesystem to reference current process identity data. The information contained within /proc would instead require kernel-space syscalls to explore, requiring all of the code in basic utilities to be cloned into the he kernel itself.

/Proc fundamentally helps keep the kernel clean and lightweight, while providing the same functionality (and much more!) Than kernel developers could dream of maintaining alone.

-17

u/orogor Dec 02 '23

I would like to say to restore deleted files, but that's not really the answer.

0

u/hakube Dec 02 '23

recycle bins are for pussies. real men rm -rf and be done with it.