100DaysOfDataEngineering Day 6,7,8 Linux

commonly used Linux commands grouped by categories.

File Management

• ls: Lists files and directories.
  • Example: ls -al (lists all files/directories with details)
• cd: Changes the current working directory.
  • Example: cd /home/username/Documents
• mkdir: Creates a new directory.
  • Example: mkdir project_folder
• rmdir: Removes empty directories.
  • Example: rmdir empty_directory
• touch: Creates an empty file or updates an existing file’s timestamp.
  • Example: touch new_file.txt
• cp: Copies files and directories.
  • Exmaple: cp important_report.doc backup_folder
• mv: Moves or renames files and directories.
  • Example: mv old_name.txt new_name.txt
• rm: Deletes files and directories.
  • Example rm unwanted_file.log

Text Manipulation

• cat: Concatenates and displays file contents.
  • Example: cat todo.txt
• less: Efficiently displays large files, allowing scrolling.
  • Example: less very_long_report.csv
• head: Displays the initial lines of a file.
  • Example: head -n 10 access.log
• tail: Displays the last lines of a file.
  • Example: tail -f error.log (monitors errors in real-time)
• grep: Searches for patterns within files.
  • Example: grep "warning" system_log.txt

System Information and Management

11. df – Display disk space usage
    • Example: df -h shows disk space in human-readable format
12. du – Estimate file space usage
    • Example: du -h shows directory space usage in human-readable format
13. top – Task manager
    • Example: top shows real-time information about running processes
14. ps – Process status
    • Example: ps -aux shows all running processes
15. kill – Kill a process
    • Example: kill 1234 terminates the process with PID 1234

File Archiving and Transfer

16. tar – Archive files
    • Example: tar -cvf archive.tar /dir creates an archive of /dir
17. gzip – Compress files
    • Example: gzip file.txt compresses file.txt to file.txt.gz
18. scp – Secure copy
    • Example: scp file.txt user@host:/path copies file.txt to the remote directory
19. wget – Network downloader
    • Example: wget http://example.com/file.txt downloads file.txt from the web
20. curl – Transfer data from or to a server
    • Example: curl http://example.com fetches the content of the specified URL

Network and Connectivity

21. ping – Check connectivity to a host
    • Example: ping google.com checks connectivity to google.com
22. netstat – Network statistics
    • Example: netstat -tuln lists active network connections
23. ifconfig – Configure network interface
    • Example: ifconfig eth0 displays configuration of eth0 interface
24. ip – Show/manipulate routing, network devices
    • Example: ip addr show lists IP addresses and network interfaces
25. ssh – Secure Shell client
    • Example: ssh user@host connects to host as user

System Administration and Maintenance

26. sudo – Execute a command as another user
    • Example: sudo apt update runs apt update as superuser
27. passwd – Change user password
    • Example: passwd changes the current user’s password
28. chown – Change file owner and group
    • Example: chown user:group file.txt changes the owner of file.txt to user
29. chmod – Change file access permissions
    • Example: chmod 755 file.txt sets file.txt permissions to 755
30. mount – Mount a filesystem
    • Example: mount /dev/sda1 /mnt mounts /dev/sda1 to /mnt

Process and Job Control

31. bg – Move a job to the background
    • Example: bg moves the current job to the background
32. fg – Move a job to the foreground
    • Example: fg brings the most recent background job to the foreground
      33.
    jobs – List active jobs
    • Example: jobs lists currently active jobs
33. nohup – Run a command immune to hangups
    • Example: nohup command & runs command immune to hangups
34. screen – Multiplex terminal, manage multiple sessions
    • Example: screen -S session_name starts a new session named session_name

Package Management

36. apt, yum, dnf – Package management commands (Debian/Ubuntu, RedHat/CentOS, Fedora)
    • Example: apt install nginx installs nginx on Debian/Ubuntu systems
37. dpkg – Debian package manager
    • Example: dpkg -l lists installed packages on Debian/Ubuntu

Text Processing

38. awk – Pattern scanning and processing language
    • Example: awk '{print $1}' file.txt prints the first column of file.txt
39. sed – Stream editor
    • Example: sed 's/old/new/' file.txt replaces ‘old’ with ‘new’ in file.txt
40. sort – Sort text files
    • Example: sort file.txt sorts the contents of file.txt alphabetically

System Services

41. systemctl – Control the systemd system and service manager
    • Example: systemctl start nginx starts the nginx service
42. service – Manage System V init services
    • Example: service nginx start starts the nginx service

Monitoring and Debugging

43. dmesg – Print kernel ring buffer
    • Example: dmesg displays messages from the kernel ring buffer
44. journalctl – Query the systemd journal
    • Example: journalctl -u nginx shows journal entries for the nginx service
45. htop – Interactive process viewer
    • Example: htop shows an interactive view of running processes

File Systems and Storage

46. fdisk – Disk partitioning tool
    • Example: fdisk /dev/sda edits the partition table for /dev/sda
47. dd – Convert and copy a file
    • Example: dd if=/dev/sda of=/dev/sdb copies /dev/sda to /dev/sdb
48. lsblk – List block devices
    • Example: lsblk lists all block storage devices

Miscellaneous Commands

49. echo – Display a line of text
    • Example: echo "Hello World" prints Hello World
50. history – Command history
    • Example: history shows the command history

This list covers a broad spectrum of frequently used Linux commands, essential for various tasks in system administration, file manipulation, networking, and more. Each command serves a specific purpose, helping users to manage and interact with the system effectively.