CSV - Comma Separated Values

-

Windows

echo Name, Age, City > data.csv

echo John Doe, 30, New York >> data.csv

echo Jane Smith, 25, Los Angeles >> data.csv


You can manipulate CSV files using Bash and various command-line tools like `awk`, `sed`, and `echo`. Below is a tutorial on how to create, write, add to, and delete from a CSV file using Bash.

### 1. Creating a CSV File

You can create a CSV file using `echo` and redirection.

```bash
#!/bin/bash

# Create a CSV file with headers
echo "Name,Age,Occupation" > data.csv
```

### 2. Writing to a CSV File

You can write to a CSV file by appending lines to it.

```bash
#!/bin/bash

# Append data to the CSV file
echo "John Doe,30,Engineer" >> data.csv
echo "Jane Smith,25,Doctor" >> data.csv
```

### 3. Adding Entries to a CSV File

Adding more entries is similar to writing. You continue appending lines.

```bash
#!/bin/bash

# Add more entries
echo "Alice Johnson,28,Artist" >> data.csv
echo "Bob Brown,45,Writer" >> data.csv
```

### 4. Reading a CSV File

You can read and display the contents of a CSV file using `cat`.

```bash
#!/bin/bash

# Display the contents of the CSV file
cat data.csv
```

### 5. Deleting Entries from a CSV File

Deleting specific lines requires a bit more work. You can use `sed` to delete lines based on their content or line number.

#### Deleting by Line Number

```bash
#!/bin/bash

# Delete the second line (Jane Smith) from the CSV file
sed -i '2d' data.csv
```
sed -i 1,3d Profiles.csv  # Selects columns 1 and 3 from file.cs

#### Deleting by Content

```bash
#!/bin/bash

# Delete lines that contain "Bob Brown"
sed -i '/Bob Brown/d' data.csv
```

### 6. Updating Entries in a CSV File

To update an entry, you can use `sed` to search and replace specific content.

```bash
#!/bin/bash

# Update "John Doe" to "John Doe,31,Senior Engineer"
sed -i 's/John Doe,30,Engineer/John Doe,31,Senior Engineer/' data.csv
```

### Putting It All Together

Here is a complete script that demonstrates creating, writing, adding, reading, deleting, and updating a CSV file:

```bash
#!/bin/bash

# Create a CSV file with headers
echo "Name,Age,Occupation" > data.csv

# Append data to the CSV file
echo "John Doe,30,Engineer" >> data.csv
echo "Jane Smith,25,Doctor" >> data.csv

# Add more entries
echo "Alice Johnson,28,Artist" >> data.csv
echo "Bob Brown,45,Writer" >> data.csv

# Display the contents of the CSV file
echo "Contents of the CSV file:"
cat data.csv
echo ""

# Delete the second line (Jane Smith) from the CSV file
sed -i '2d' data.csv

# Display the contents after deletion
echo "Contents after deleting the second line:"
cat data.csv
echo ""

# Delete lines that contain "Bob Brown"
sed -i '/Bob Brown/d' data.csv

# Display the contents after deletion
echo "Contents after deleting lines containing 'Bob Brown':"
cat data.csv
echo ""

# Update "John Doe" to "John Doe,31,Senior Engineer"
sed -i 's/John Doe,30,Engineer/John Doe,31,Senior Engineer/' data.csv

# Display the final contents of the CSV file
echo "Final contents after update:"
cat data.csv
```

Save this script to a file (e.g., `csv_script.sh`), make it executable, and run it:

```bash
chmod +x csv_script.sh
./csv_script.sh

5. Converting CSV Format (csvformat)
csvformat -T example.csv > example_tab_delimited.csv

6. Aggregating Data (csvstat with --sum, --mean, etc.)

csvstat --sum -c Amount example.csv

--sum, --mean, --median, --mode, --max, --min)

7. Joining CSV Files (csvjoin)

# Inner join two CSV files on 'ID' column
csvjoin -c ID file1.csv file2.csv

8. Sorting Data (csvsort)

# Sorting CSV data by 'Date' column in ascending order
csvsort -c Date example.csv



Table of Contents

  1. Introduction to CSV
  2. Working with CSV in Python
    • Using csv module
    • Using pandas
  3. Command-Line Tools for CSV
    • CSVKit - Installation - pip install csvkit
    • csvlook example.csv - read the csv file.
    • csvgrep -c Country -m "USA" example.csv - specific row or data or column
    • csvstat example.csv - Statictics
    • csvcut -c Name,Age example.csv - select specific columns
    • AWK and Sed
  4. CSV Operations in Excel / Google Sheets
  5. Best Practices for CSV Handling
  6. Additional Resources
```

This script demonstrates how to create a CSV file, add entries, read the contents, delete specific lines, and update existing entries using Bash and common command-line tools.

#csv
#datamanagement

Comments

Post a Comment

Popular posts from this blog

SAMBA SERVER 2.0 #server

-
 To share a folder on a LAN using Samba on a Linux system, follow these steps: 1. **Install Samba**: If it's not already installed, you need to install Samba. Open a terminal and run:    ```bash    sudo apt update    sudo apt install samba    ``` 2. **Create a Directory to Share**: Create the directory you want to share, if it doesn't exist. For example:    ```bash    sudo mkdir -p /srv/samba/share    sudo chmod 2775 /srv/samba/share    sudo chown nobody:nogroup /srv/samba/share    ``` 3. **Configure Samba**: Edit the Samba configuration file to add your share.    Open the Samba configuration file in a text editor:    ```bash    sudo nano /etc/samba/smb.conf    ```    Add the following configuration at the end of the file:    ```ini    [share]    path = /srv/samba/share    browseable = yes    read only = no    guest ok = yes    create mask = 0660    directory mask = 2770    force user = nobody    force group = nogroup    ```    This configuration sets up a share called "sh
Read more

Setup SSH for accessing outside from network

-
To set up port forwarding for SSH from a router to allow external access to a Kali Linux machine, you need to configure your router and ensure your Kali Linux SSH server is correctly set up. Here's a step-by-step guide on how to do this: ### Step 1: Configure SSH on Kali Linux 1. **Install OpenSSH Server** (if not already installed):    ```bash    sudo apt update    sudo apt install openssh-server    ``` 2. **Start and Enable SSH Service**:    ```bash    sudo systemctl start ssh    sudo systemctl enable ssh    ``` 3. **Check SSH Status**:    ```bash    sudo systemctl status ssh    ``` 4. **Optional: Change Default SSH Port** (for security):    Edit the SSH configuration file:    ```bash    sudo nano /etc/ssh/sshd_config    ```    Change the line:    ```    #Port 22    ```    to something like:    ```    Port 8022    ```    Save the file and restart the SSH service:    ```bash    sudo systemctl restart ssh    ``` ### Step 2: Configure Port Forwarding on the Router Since router inter
Read more

Speech tools - espeak and festival etc.

-
 Using speech tools in Kali Linux involves installing and utilizing various software packages designed for speech processing, synthesis, and recognition. Here are the steps to get you started: 1. Install Required Packages Kali Linux is based on Debian, so you can use apt to install the necessary packages. Some popular speech tools include espeak , festival , and pocketsphinx . Install espeak espeak is a compact open-source software speech synthesizer for English and other languages. sudo apt update sudo apt install espeak You can test espeak by running: espeak "Hello, this is a test of the espeak system." Install festival festival is a general-purpose, multi-lingual speech synthesis system. sudo apt install festival You can test festival by running: echo "Hello, this is a test of the festival system." | festival --tts Install pocketsphinx pocketsphinx is a lightweight speech recognition engine from CMU Sphinx. sudo apt install pocketsphinx pocketsphinx-en-us Yo
Read more