How to parse .csv with Python

How to parse .csv with Python

Science + Python = Profit!

Python in Science

Python ๐Ÿ is a great language for beginners ๐Ÿ‘ถ (I myself started coding with it). It's especially useful for creating small tasks โš™ on your computer, automating file management ๐Ÿ“‚, calculations ๐Ÿงฎ, document formatting ๐Ÿ“„...

So you won't be surprised to know that it's the preferred language among scientists ๐Ÿ‘ฉ๐Ÿปโ€๐Ÿ”ฌ. A lot of artificial intelligence libraries are written in Python, but other science fields like Chemistry ๐Ÿงช use Python in their calculus programs, like the R project.

Solving a problem

Let me share a little example with you. One day, a chemist (who happens to be also my wife ๐Ÿ˜‡) asked me how to transform a concrete .csv ๐Ÿ“ƒ columns and rows structure into a different structure automatically.

The UV-vis spectrophotometer machine โš™ my wife was using at the university laboratory was outputting a format "A", and she needed a format "B" as input for the analysis software ๐Ÿ‘จ๐Ÿผโ€๐Ÿ’ป(ReactLab JPlus Consulting) that was processing the scientific data.

How to decide when to automate a task

She has been doing that task manually for a week ๐Ÿ“…, realizing that all that time was being wasted on something that should be trivial. Time โณ that she could have been investing in doing actual scientific work.

In addition to that, she wasn't the only one who had to suffer this manual process: the machine was used by other colleagues ๐Ÿฅผ in the present, and potentially, in the future.

This kind of situation is ideal for creating a task and make somebody more productive and probably happier ๐Ÿ™‚. To be sure about it, just sum the hours ๐Ÿ•— each person is wasting on a task daily, and multiply it by all the persons ๐Ÿ‘จโ€๐Ÿ‘จโ€๐Ÿ‘งโ€๐Ÿ‘ง in that situation (and consider persons outside the organization that could benefit from it too! ๐ŸŒ). If the resulting number is bigger than the hours of effort ๐Ÿ’ฆ you will need to invest, it's generally a good idea to develop that task โœ….

So I asked my wife ๐Ÿ’๐Ÿฝโ€โ™€๏ธ an example for the input.csv and the output.csv and started working!

Some details

Python includes by default:

1- A csv module ๐Ÿงฉ that gave me the feature I needed to write the output.csv.

2- A glob module ๐Ÿงฉ that allowed me to read the input.csv

So, I just needed to develop the logic ๐Ÿฆพ for rearranging and restructuring the columns and row!

Finally, I just wrote a small batch file that could be clicked ๐Ÿ–ฑ by the user so they didn't need to use their OS terminal.

Closing time

I named the task "Bondy" as a reference to the concept of covalent bonds โ›“ in chemistry.

I tried to keep the README very simple so people with no programming experience could use the ask without issues ๐Ÿ˜Š. The harder part of the setup was installing Python, luckily MacOS includes it by default and Windows users can just download and install it without needing more than user-level computer knowledge ๐Ÿ˜Ž...

This task helped to publish this research paper ๐Ÿ“ƒ, which makes me quite happy as an old hard-core Science supporter ๐Ÿ’“

If you are a scientist ๐Ÿ‘จ๐Ÿฟโ€๐Ÿ”ฌ, I think you could benefit a lot from learning a bit of Python to create your own small tasks. If you combine that skill with some spreadsheets ๐Ÿ“„ skills, you will be able to work faster and be more independent. Free your mind from repetitive work, learn to code today!

Did you solve any similar problems with some lines of code? Did it blow your mind ๐Ÿคฏ? Let me know in the comments ๐Ÿ˜‰.

ย