Skip to main content
DevOps Insightsguides & answers

DevOps · beginner

Linux Commands Every DevOps Engineer Must Know

Short answer

You do not need to memorise a hundred commands. You need about fifteen, and you need to know which question each one answers. This page is organised by the question, because that is how it arrives at three in the morning — not as "I would like to use lsof now".

N
Neeraj Jha
Updated 7 min read485 views
On this page · 13 sections
Terminal showing ss, journalctl, df and du, ps and dmesg commands, ending with a kernel out-of-memory line.

Who this is for

You can SSH into a box and read a prompt. You are not trying to become a sysadmin; you are trying to find out why something is broken and get back to what you were doing.

"The box is slow"

bash
uptime            # load average: 1, 5 and 15 minute
top               # or htop, if it is installed
free -h           # memory

Load average on Linux is not CPU utilisation. It counts processes that are running and processes stuck in uninterruptible sleep — almost always waiting on disk or network storage. A load of 8 on an 8-core box can mean fully busy, or it can mean eight processes stuck waiting on a slow disk while every core idles. top tells you which: look at %wa (IO wait) against %us (user).

In free -h, read the available column, not free. Linux spends spare memory on page cache deliberately and gives it back on demand — a box showing almost no "free" memory and plenty "available" is working correctly. This is a classic false alarm on a Linux server.

"Is it really short of memory?"

Added 26 October 2026. The available column says how much memory is left. It does not say whether the box is struggling; that shows as activity:

bash
vmstat 5                      # si and so: pages swapped in and out per second
cat /proc/pressure/memory     # share of time tasks waited for memory (on kernels with PSI)

Two readings from this site's own 2 GB server, measured on 8 October 2026. On a normal day, free showed 94 MB free but 1,363 MB available, 54 MB of swap in use, no swapping in progress and memory pressure at zero: healthy, however alarming the free column looks. During a Next.js build, available fell to 269 MB, swap in use passed 1.1 GB and pages moved in and out of swap at tens of megabytes a second: that is pressure. Swap in use on its own is history, not a problem.

Do not "fix" low free memory with echo 3 > /proc/sys/vm/drop_caches. The kernel documentation says those caches are reclaimed automatically when memory is needed, and that dropping them costs I/O and CPU to rebuild. The comparison in full: free vs available memory.

"The disk is full"

bash
df -h                 # which filesystem, and how full
du -sh /var/* | sort -rh | head    # where it went, under a path

When df says the disk is full and du cannot account for it, the space is almost always held by files that have been deleted but are still open. A process holding a deleted log file keeps its blocks allocated until the process exits — the directory entry is gone, so du cannot see it, but the space is still spent.

bash
sudo lsof +L1     # open files whose link count is 0

Restart whatever shows up, or truncate it. Deleting "the big log file" again achieves nothing, which is why this one wastes so much time when you have not seen it before.

"The service will not start"

bash
systemctl status nginx            # state, and the last few log lines
journalctl -xeu nginx             # the actual reason, with explanations
journalctl -u nginx --since "10 min ago" --no-pager

systemctl status shows a handful of recent lines and people stop there. The cause is usually further back, which is what journalctl -u is for. -x adds explanatory text for systemd's own messages, and -e jumps to the end.

"What is listening on that port?"

bash
sudo ss -tulnp                    # every listening socket, with the process
sudo lsof -i :8080                # who holds one specific port

Run both with sudo. Without it they still list the sockets but omit the process names, which is the part you wanted — a surprisingly common ten minutes lost.

"Can this box reach that one?"

bash
curl -v https://example.com       # the whole exchange: DNS, TLS, headers
curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' https://example.com
dig example.com +short            # what DNS actually returns
nc -zv hostname 443               # is the port open at all
ip route get 10.0.1.5             # which interface and gateway it would use

Work down the stack. dig first — if the name resolves to the wrong address, nothing above it matters. Then nc for reachability, then curl -v for TLS and HTTP. Skipping to curl and reading the error message is how a DNS problem gets diagnosed as a certificate problem.

"Find it in the logs"

bash
grep -c "ERROR" app.log                       # how many, before you read any
grep -n "timeout" app.log | tail -20          # with line numbers
awk '$9 >= 500 {print $7}' access.log | sort | uniq -c | sort -rn | head
sed -n '2000,2050p' huge.log                  # a slice, without loading it all
tail -f /var/log/syslog                       # follow (journalctl -f where there is no syslog file)

The sort | uniq -c | sort -rn pipeline is the one to internalise: count occurrences, then rank them. It turns "a lot of errors" into "97% of them are one error", which is a different problem.

For compressed and rotated logs, zgrep reads them without unpacking first.

Permissions, briefly

bash
chmod +x deploy.sh                      # make it runnable
chmod 644 config.yml                    # rw for owner, r for everyone else
chown -R www-data:www-data /var/www
id                                      # who am I, and in which groups
sudo -u postgres psql                   # run as another user

If a script fails with "Permission denied" and the file looks executable, check the directory too — you need execute permission on every directory in the path to reach the file.

Common mistakes

  1. Reaching for kill -9 first. -9 is SIGKILL: the process cannot catch it, so it never flushes a buffer, closes a connection or removes a lock file. Send kill (SIGTERM) first and give it a few seconds; keep -9 for something that has genuinely stopped responding.
  2. base64 without -n. echo "secret" | base64 encodes a trailing newline, and the decoded value will not match. echo -n matters, and this is the usual reason a hand-made Kubernetes secret authenticates nowhere.
  3. chmod 777 to make it work. It does make it work, and it makes it world-writable. The problem is nearly always ownership, which chown fixes without the side effect.
  4. find / -name ... on a production box under load. It walks every filesystem, including network mounts. Give it a starting path and -xdev to stay on one filesystem.

If this did not work

  • A command is not found. Minimal images ship very little — htop, dig, lsof and netstat are frequently absent. ss and ip are usually there; cat /proc/net/tcp always is.
  • You are inside a container and the numbers look wrong. top and free read the host's /proc, so they report the host's memory and CPU, not the container's limits. Read /sys/fs/cgroup/ for what the container is actually allowed.
  • journalctl returns nothing. The service may not be running under systemd, or logs may be going to a file. Check the unit's StandardOutput, or look in /var/log/.

Next steps

Practise the disk-full and the port-in-use ones on a throwaway box before you need them. Both come up in real incidents, and both have a counter-intuitive answer that is hard to reason out under pressure.

Tagged with

Enjoyed this article?

Get more DevOps insights delivered to your inbox.

Get new posts by email

Subscribe to get an email when a new blog post is published. Skip anytime.

No spam, unsubscribe anytime.

N

Written by

Neeraj Jha

Platform administrator and lead writer.

View all posts →
Terms used here (2)

Short definitions for the vocabulary on this page, in case any of it is new. Each links to the full entry.

Ansible
An open-source automation tool for configuration management, application deployment, and task automation. Ansible uses YAML-based playbooks and operates agentlessly over SSH, making it simple to get started without installing software on target machines.
SSH
Secure Shell — a cryptographic network protocol for operating network services securely over an unsecured network. SSH is the standard method for remotely accessing Linux servers, transferring files, and tunneling other protocols.

All 59 terms in the glossary

Discussion

0 comments

Sign in to join the conversation.

Be the first to comment

Start a conversation about this post

Share: