CSV - Comma Separated Values

on

Windows

echo Name, Age, City > data.csv

echo John Doe, 30, New York >> data.csv

echo Jane Smith, 25, Los Angeles >> data.csv


You can manipulate CSV files using Bash and various command-line tools like `awk`, `sed`, and `echo`. Below is a tutorial on how to create, write, add to, and delete from a CSV file using Bash.

### 1. Creating a CSV File

You can create a CSV file using `echo` and redirection.

```bash
#!/bin/bash

# Create a CSV file with headers
echo "Name,Age,Occupation" > data.csv
```

### 2. Writing to a CSV File

You can write to a CSV file by appending lines to it.

```bash
#!/bin/bash

# Append data to the CSV file
echo "John Doe,30,Engineer" >> data.csv
echo "Jane Smith,25,Doctor" >> data.csv
```

### 3. Adding Entries to a CSV File

Adding more entries is similar to writing. You continue appending lines.

```bash
#!/bin/bash

# Add more entries
echo "Alice Johnson,28,Artist" >> data.csv
echo "Bob Brown,45,Writer" >> data.csv
```

### 4. Reading a CSV File

You can read and display the contents of a CSV file using `cat`.

```bash
#!/bin/bash

# Display the contents of the CSV file
cat data.csv
```

### 5. Deleting Entries from a CSV File

Deleting specific lines requires a bit more work. You can use `sed` to delete lines based on their content or line number.

#### Deleting by Line Number

```bash
#!/bin/bash

# Delete the second line (Jane Smith) from the CSV file
sed -i '2d' data.csv
```
sed -i 1,3d Profiles.csv  # Selects columns 1 and 3 from file.cs

#### Deleting by Content

```bash
#!/bin/bash

# Delete lines that contain "Bob Brown"
sed -i '/Bob Brown/d' data.csv
```

### 6. Updating Entries in a CSV File

To update an entry, you can use `sed` to search and replace specific content.

```bash
#!/bin/bash

# Update "John Doe" to "John Doe,31,Senior Engineer"
sed -i 's/John Doe,30,Engineer/John Doe,31,Senior Engineer/' data.csv
```

### Putting It All Together

Here is a complete script that demonstrates creating, writing, adding, reading, deleting, and updating a CSV file:

```bash
#!/bin/bash

# Create a CSV file with headers
echo "Name,Age,Occupation" > data.csv

# Append data to the CSV file
echo "John Doe,30,Engineer" >> data.csv
echo "Jane Smith,25,Doctor" >> data.csv

# Add more entries
echo "Alice Johnson,28,Artist" >> data.csv
echo "Bob Brown,45,Writer" >> data.csv

# Display the contents of the CSV file
echo "Contents of the CSV file:"
cat data.csv
echo ""

# Delete the second line (Jane Smith) from the CSV file
sed -i '2d' data.csv

# Display the contents after deletion
echo "Contents after deleting the second line:"
cat data.csv
echo ""

# Delete lines that contain "Bob Brown"
sed -i '/Bob Brown/d' data.csv

# Display the contents after deletion
echo "Contents after deleting lines containing 'Bob Brown':"
cat data.csv
echo ""

# Update "John Doe" to "John Doe,31,Senior Engineer"
sed -i 's/John Doe,30,Engineer/John Doe,31,Senior Engineer/' data.csv

# Display the final contents of the CSV file
echo "Final contents after update:"
cat data.csv
```

Save this script to a file (e.g., `csv_script.sh`), make it executable, and run it:

```bash
chmod +x csv_script.sh
./csv_script.sh

5. Converting CSV Format (csvformat)
csvformat -T example.csv > example_tab_delimited.csv

6. Aggregating Data (csvstat with --sum, --mean, etc.)

csvstat --sum -c Amount example.csv

--sum, --mean, --median, --mode, --max, --min)

7. Joining CSV Files (csvjoin)

# Inner join two CSV files on 'ID' column
csvjoin -c ID file1.csv file2.csv

8. Sorting Data (csvsort)

# Sorting CSV data by 'Date' column in ascending order
csvsort -c Date example.csv



Table of Contents

  1. Introduction to CSV
  2. Working with CSV in Python
    • Using csv module
    • Using pandas
  3. Command-Line Tools for CSV
    • CSVKit - Installation - pip install csvkit
    • csvlook example.csv - read the csv file.
    • csvgrep -c Country -m "USA" example.csv - specific row or data or column
    • csvstat example.csv - Statictics
    • csvcut -c Name,Age example.csv - select specific columns
    • AWK and Sed
  4. CSV Operations in Excel / Google Sheets
  5. Best Practices for CSV Handling
  6. Additional Resources
```

This script demonstrates how to create a CSV file, add entries, read the contents, delete specific lines, and update existing entries using Bash and common command-line tools.

#csv
#datamanagement

Comments

Post a Comment