💡 This article assumes that you have a basic understanding of the HTTP protocol, that you know what a web API is, and that you are familiar with the CLI of your operating system.

Table of contents:

• ⚙The setup
• 🧩 Basic curl options
• 📩 The GET method
• 📮 The POST method
  • 🔤 Sending plain text
  • 🖇 Query string parameters
  • 💌 Sending a JSON object
  • 📋 Emulating a form
• 🎯 The other methods
• 🔐 Authentication
• 🤹‍♂️ Getting creative

⚙The setup

Well, just like almost every piece of software out there, curl needs to be installed on your operating system. The good news is that most likely, your system already comes with it built-in. Almost every Linux distribution comes with curl and Windows 10 also started shipping it from version 1803, too 😉.

Would you like to check if your system has it? Well, let's give it the very first test drive.

On Windows, you may try the following command either on CMD or PowerShell:

curl.exe --version

On Linux, you may try this on your terminal:

curl --version

The command should return some information about the curl version, libraries it counts with, release date, supported protocols and features. I'm a Linux user and in my case, I got the following output:

curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

If you get an error instead, you can also download and install curl. It's free! It's available for all major operating systems out there and last time I checked, it only takes a ~5MB download.

If you're exploring this article but don't currently have an API handy to play with, not to worry. The httpbin service is used in the examples of this post 😉.

💡 Note: From this point on, bash commands will be given as examples for clarity and readability. If you are using Windows or the example snippets don't work for you, try appending .exe after the curl command or try removing the line breaks.

🧩 Basic curl options

Using curl is pretty simple and though it can handle a plethora of different options, you might want to start by getting familiar with the following ones:

• --request or -X: To specify the desired HTTP request method. If the option is not present, curl will treat the request as GET. Example: curl -X POST ...
• --header or -H: To include an extra header on the HTTP request. Any number of extra headers can be included. Example: curl -H "X-First-Name: John" -H "X-Last-Name: Doe" ...
• --data or -d: To include data on the HTTP request. The data can be either included on the same command line or it can reference a text-based file to read the data from. Example: curl -d "text data goes here" ...
• --form or -F: Lets curl emulate a filled-in form in which a user has pressed the submit button. This also enables uploading of binary files. Example: curl -F name=John -F shoesize=11 ...
• --user <user:password> or -u <user:password>: Specify the username and password to use for server authentication.

curl can present the whole HTTP request transaction with great detail in the terminal, however, I prefer it when it dumps the results into files for further review. To achieve this, you can make use of these options:

• --dump-header or -D: Write the received protocol headers to the specified file. Example: curl -D result.header ...
• --output or -o: Write output to the specified file instead of stdout. Example: curl -o result.json ...

📩 The GET method

Well, now that you are ready with the basics, it's time to GET the party started! 🥳

First, I'd recommend you to change the working directory on your CLI to one that you're familiar with such as your user directory or your desktop so that you're able to easily find the resulting files after executing the commands.

Let's make our very first request with:

curl -X GET "https://httpbin.org/get" \
-H "accept: application/json" \
-D result.headers \
-o result.json

On the first line, we're telling curl to perform an HTTP request with the GET method to the specified URL. On the second line, we add a header to let the server know what we'll accept as a response. On the third line, we tell curl to dump the resulting headers of our request to a file called result.headers. Lastly, we tell curl to write the output of the request to a file called result.json.

If everything went nice and smoothly, you should be able to see the resulting files in your working directory after executing the command. In my case, I got:

result.headers

HTTP/2 200 
date: Thu, 02 Sep 2021 05:47:14 GMT
content-type: application/json
content-length: 169

In the result.headers file, we can see that the request went through successfully with a 200 HTTP response code, the response timestamp and all headers returned by the server.

result.json

{
  "args": {}, 
  "headers": {
    "Accept": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.68.0"
  },
  "url": "https://httpbin.org/get"
}

The result.json file contains the body of the response. In this case, the data that gets returned by the server following the JSON notation.

Pretty neat, don't you think?

📮 The POST method

🔤 Sending plain text

This time, let's try sending a plain text message to the server by using the POST method.

curl -X POST "https://httpbin.org/post" \
-H "accept: application/json" \
-H "Content-Type: text/plain" \
-H "Custom-Header: Testing" \
-d "I love hashnode" \
-D result.headers \
-o result.json

Note that we've included the header "Custom-Header: Testing". It should get reflected in the body of the response.

result.headers

HTTP/2 200 
date: Fri, 03 Sep 2021 03:16:20 GMT
content-type: application/json
content-length: 182

result.json

{
  "data": "I love hashnode", 
  "headers": {
    "Accept": "application/json", 
    "Content-Length": "15", 
    "Content-Type": "text/plain", 
    "Custom-Header": "Testing"
  }
}

Did you notice that the server received the message along with the testing header and echoed them? That means curl was able to send the data we wanted it to.

🖇 Query string parameters

Not only sending plain text is supported by curl but also query string parameters are supported:

curl -X POST "https://httpbin.org/post?name=Carlos&last=Jasso" \
-H "accept: application/json" \
-D result.headers \
-o result.json

result.headers

HTTP/2 200 
date: Fri, 03 Sep 2021 03:30:47 GMT
content-type: application/json
content-length: 120

result.json

{
  "args": {
    "lastname": "Jasso", 
    "name": "Carlos"
  }, 
  "headers": {
    "Accept": "application/json"
  }
}

Whoa! The server echoed the parameters curl sent to it over the query string.

💌 Sending a JSON object

Let's give curl another task and make it send a json file to the server and see what happens:

curl -X POST "https://httpbin.org/post" \
-H "Content-Type: application/json; charset=utf-8" \
-d @data.json \
-o result.json

This time, we're including a header to let the server know we're sending a json file. The data file path gets specified to curl with the -d option followed by an at sign @ and the path to the file.

Here's the content of the data.json file for you to create it on your working directory:

{
    "name": "Jane",
    "last": "Doe"
}

result.json

{
  "data": "{    \"name\": \"Jane\",    \"last\": \"Doe\"}", 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "38", 
    "Content-Type": "application/json; charset=utf-8"
  }, 
  "json": {
    "last": "Doe", 
    "name": "Jane"
  }
}

Yet again, curl was able to send the JSON file content to the server as we can see the data echoed on the response. Kudos for curl!

📋 Emulating a form

Sometimes, you might want to emulate submitting a form and curl can make that possible:

curl -X POST "https://httpbin.org/post" \
-H "Content-Type: multipart/form-data" \
-F "FavoriteFood=Pizza" \
-F "FavoriteBeverage=Beer" \
-o result.json

For curl to specify to the server that what's being sent is form data, you can use a request header with the appropriate form MIME type as seen above and use the -F parameter for each of the form fields and values.

result.json

{ 
  "form": {
    "FavoriteBeverage": "Beer", 
    "FavoriteFood": "Pizza"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "261", 
    "Content-Type": "multipart/form-data;"
  }
}

In the result, we can confirm that the server received a form and interpreted each of its fields correctly.

🖼 Sending a file

So far, we've given curl lightweight work. How about sending a file on this round? Files are sent in a similar way forms are emulated. Would you like to copy any image to your working directory and try to send it?

curl -X POST "https://httpbin.org/post" \
-H "Content-Type: multipart/form-data" \
-F "FileComment=This is a JPG file" \
-F "image=@image.jpg" \
-o result.json

result.json

{
  "files": {
    "image": "data:image/jpeg;base64,/9j/4AAQSkZJ..."
  }, 
  "form": {
    "FileComment": "This is a JPG file"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "78592", 
    "Content-Type": "multipart/form-data;"
  }
}

It might take some time for the file to upload but in the end, the server will return the same file we sent to it (base64 encoded). curl has correctly handled every request we've built with it so far.

🎯 The other methods

I've tested as many HTTP request methods as I've been able to think of with curl and all of them have worked splendidly. After all, what the API does with each request may vary depending on the implementation of the method.

🔐 Authentication

curl can handle server authentication when accessing a URL that requires the user to input credentials. In this case, httpbin receives the expected username and password through the URL with the format /basic-auth/{user}/{passwd} to match the values with the inputted credentials. This is what happens if said credentials are not entered:

curl -X GET "https://httpbin.org/basic-auth/carlos/secret" \
-H "accept: application/json" \
-D result.headers

result.headers

HTTP/2 401 
date: Fri, 03 Sep 2021 04:08:44 GMT
content-length: 0

Dang! got a 401 (unauthorized) 🤔

Oh right, we forgot to include our credentials. Let's include them in the options:

curl -X GET "https://httpbin.org/basic-auth/carlos/secret" \
-u carlos:secret
-H "accept: application/json" \
-D result.headers

result.headers

HTTP/2 200 
date: Fri, 03 Sep 2021 04:14:19 GMT
content-type: application/json
content-length: 48

result.json

{
  "authenticated": true, 
  "user": "carlos"
}

Yes! got logged in 😎.

Another common authentication method is bearer tokens but they tend to be added within a header and we have already covered that case so, I bet an example for those can be skipped.

🤹‍♂️ Getting creative

How have you liked curl so far? Do you think it's useful? Wait until I let you know about some extra suggestions I think you might also like.

I've seen that common API testing tools are free as well, but they put price tags on their collaboration features and that's fine. That's how they raise funds to keep their motors running but people like me who prefer free products and services might want to look somewhere else. What I like to do is to have API documentation on markdown files on a git repo like this one and share them with other people. To me, that enables collaboration with curl without having to spend a single cent.

Also, other tools offer sophisticated GUIs but I find most of them to be too cluttered. I like simplicity and if you do, too, I'd recommend using VSCode if you're not using it already. I think VSCode + curl just make a perfect match ❤

That's the workflow I normally use. You can see the rendered markdown document to the left, my result.headers & result.json to the right and a terminal at the bottom to enter all my pre-designed commands. What do you think about it?

Do you think curl can only work fine with the REST architecture? well, I've also had the chance to use it to test SOAP with the same great results. I'm just waiting to have a new scenario to put curl to the test 🤓.

I hope this post has been both fun and useful to you. Now, you know how to take advantage of curl for your API testing purposes plus you'll also look like those hackers shown on TV when you're using your terminal and text-based tools at work. If that isn't cool, I don't know what is.

If you'd like to become a curl guru and keep learning about it, I'd encourage you to visit its documentation page. There's really a vast number of different scenarios it can be used for.

Fun fact: I don't know how to use Postman.

👋

_{^{Cover Credits: Computer vector created by upklyak - www.freepik.com | https://commons.wikimedia.org/wiki/File:Curl-logo.svg, curl contributors, MIT http://opensource.org/licenses/mit-license.php, via Wikimedia Commons}}

⚠ Disclaimer: This article does not provide a CORRECT nor a SUBLIME way of doing things. Instead, it provides tips based on the writer's experience.

🌱 The root of the file tree

In order to end up with a usable project file structure, this article will guide you on the definition of one meant to be used by a react application. We'll begin with the very root of the directory assuming that an NPM package has already been initiated. It should look like this:

.                   <-- represents the directory root
└── package.json    <-- NPM package definition

It's so clean and perfect to this point but extra stuff needs to go inside of it. This root of the project's directory should be used to allocate files that might not be totally necessary for the final project's product but for the developer to know that to work with and tools to know how to act like. Here's a list of example files one can commonly find in this location:

• 📃 README - Documentation about the project itself. It may contain the purpose of the project, description of its functionality, documentation on build steps, etc.
• ⚖ LICENSE - Specifies how the project can be used or distributed along with its constraints to do so.
• 🙅‍♂️ IGNORE - This is commonly found when there's a version control system set on the project. If Git is being used, the file .gitignore could be included.
• ⚙ CONFIG - Depending on the project you're building, you might find tools configurations on the root directory, too. For example, if the project is a .net application, it's common to find an .sln file, if the project is a nodejs application using typescript, a bundler such as webpack and the documents get automatically formatted by Prettier, you may find files such as tsconfig.json, webpack.config.js and .prettierrc respectively.
• ♻ ENVIRONMENT - Some projects may rely on environment variables. Most of the times, these variables are defined in the root directory. For example, the Azure Functions Core Tools rely on a local.settings.json file for this purpose while NodeJS relies on .env files instead.

A root directory hosting some the aforementioned examples may look similar to this:

.
├── .env.development
├── .gitignore
├── LICENSE
├── README.md
├── package.json
├── tsconfig.json
└── webpack.config.js

As you may notice, none of the files are really needed by a project's final product but they are important for a developer to work on the project. Any other files that have similar purposes should be stored on the root the project's directory to avoid losing track of them or get them mixed up with the project's actual source code.

🗃 Project sub-directories

It's common to find several sub-directories in a project, and each of them serves a different purpose. It totally depends on the kind of project you're building but usually one can find a structure similar to the following:

• ⚙ .DIRECTORIES - Many tools depend on more than a single configuration file and instead of storing a dot-file on the root of the project's directory, they may create a sub-directory with its name starting with a "." (dot) to store their configurations in a single place. Good examples you may be familiar with are VSCode and Git bucease if you're making use of them, you for sure can find the .vscode and .git directories in your project.
• 🤹‍♂️ ASSETS - Contains files that may not need modifications and can be simply copied as part of the project's resulting output. Common files that are stored here are favicons, avatar pictures, type fonts, manifests, re-distributable scripts, etc. Common names used for this directory are assets, resources and static.
• 📚 DOCUMENTATION - You may know every bolt and nut holding your project together but when you plan to share it with other developers, it's a very good practice to let them know how the project works, looks, gets used and gets built via written documentation. Frequently, this directory is named docs.
• 🍔 OUTPUT - The project gets compiled and its resulting product is usually put apart from the source code in a specific directory so that it can be easily grabbed. Usual names for this directory can be out, result, public, dist and bin. Also, it's customary to exclude this directory from source version control systems like Git.
• 👨‍💻 SOURCE CODE - Here's where the golden nuggets, the jewels, the juicy stuff gets stored at. More to come about this special directory below 😉. An habitual name for this directory is src.
• 🧪 TESTS - Another good practice among developers is to create and share automated tests for our project. This helps ensure that what the project is aimed to do is actually what it does. Most tests are written code, too. But mixing the tests code with the source code can crate confusion, so, it's better to separate them and give them their own place. test is a very common name for this directory.

Here's an example of a project directory containing some of the sections mentioned above:

.
├── assets
│   ├── avatar.jpg
│   └── favicon.ico
├── docs
│   ├── README.md
│   ├── api-reference.md
│   └── build.md
├── out
├── src
│   ├── app.tsx
│   ├── footer.tsx
│   └── nav.tsx
├── test
│   ├── app.test.js
│   └── nav.test.js
├── .env.development
├── .gitignore
├── LICENSE
├── README.md
├── package.json
├── tsconfig.json
└── webpack.config.js

👨‍💻 Source Code file structure

🥪 The layered approach

Not all applications will have the same structure, but they all can be divided into "concern layers." For example, when developing an API, you may want to include all controllers into their own directory with a separate business logic within a services directory and data definitions within a models directory. This to aim for a better understanding of the project structure, scalability and ease of maintenance.

Another example is a desktop application wherein a UI directory may hold all window designs but a services directory would feature all business logic, a models directory could hold all data definitions and a data directory all database transactions logic.

With this approach in mind, we can elaborate our src directory on our structure and add a pages directory for entities that will make use of several modules at a time, a layouts directory for pages "themes", a modules directory where entities of the app are defined such as a header, welcome, posts list, footer, etc. All these make the src directory look similar to this:

src
├── layouts
├── models
├── modules
│   ├── footer
│   ├── header
│   └── post-list
└── pages

💎 SOLID principles

These principles mainly apply to Object Oriented Programming but taking them as guidance will also give you a nice result with other methodologies.

• Single responsibility: A class should only have a single responsibility. That is, is should only affect a single specification of a program.
• Open-Closed principle: Software entities such as classes, modules, functions, etc. Should be open to be extended in functionality without having to be modified at their source code level.
• Liskov substitution principle: Taking S as a subtype of T, objects of type T may be replaced by objects of type S without the program suffering a change on its desired properties.
• Interface segregation principle: Have you ever had a bad experience using a "universal" remote control? Well, the same thing happens with programming; a remote control may be thought like an interface for controlling a device such as a TV. The interface defines the actions that a recipient of it can do. This principle states that a recipient or client should not be forced to feature an action it's not going to use, therefore, it's always better to have several small interfaces rather than a big one.
• Dependency inversion principle: The summary of this principle is "High level objects should not depend on low level implementations." This is, any high-level entity in a program should be able to work under several scenarios and its functionality should be dependent of low-level implementations. For example, a "User Administration" module should be able to take care of its business without being dependent of a "User" class where features such as name, age, registration date or other data is defined. This same module should be able to leverage the task of saving data to a database by receiving the indication to do so, but the actual task should be delegated to a low-level class that takes care of saving the information.

More about this here

Knowing this, we can then think of adding files to the src directory sub-paths like this:

src
├── layouts
│   ├── blog-layout.tsx
│   ├── common-layout.tsx
│   └── home-layout.tsx
├── models
│   ├── blog-post.ts
│   └── navbar-item.ts
├── modules
│   ├── footer
│   │   ├── clickable-icon.tsx
│   │   ├── external-links.tsx
│   │   ├── index.tsx
│   │   └── site-sections.tsx
│   ├── header
│   │   ├── index.tsx
│   │   ├── logo.tsx
│   │   ├── menu-button.tsx
│   │   ├── nav-button.tsx
│   │   └── navbar.tsx
│   └── post-list
│       ├── index.tsx
│       ├── list.tsx
│       └── post-card.tsx
└── pages
    ├── about.tsx
    ├── blog.tsx
    ├── error404.tsx
    └── home.tsx

I hope you've enjoyed this article. Now, you have a couple of guidance points to follow when having to design a project's file structure.

^{_{Cover credits: Diagram made with diagrams.net | Banner vector created by pch.vector - www.freepik.com | FiraCode}}

💡 This article assumes that both Node and NPM are already installed on your operating system. If not, you might want to head up to this other article before you continue on this one.

Table of contents:

• 📁 Create a directory for the app to live in
• 📦 Initialize the NPM package
• 🔬 Anatomy of the package.json file
• 🤓 Taking a shortcut
• 🧩 Dependency version specification

📁 Create a directory for the app to live in

The package will hold everything for your app. This includes: source code, configurations, assets and even documentation. Everything co-exists in a directory together, and it can be manually created. You may use your favorite directory creation method. The only recommendation for you is not to use spaces on the directory's name. Why? well, this will make it easier to navigate and prevent errors in some scenarios. Once the directory is created, go inside of it:

mkdir react-from-scratch && cd react-from-scratch

📦 Initialize the NPM package

An NPM package depends a lot on a very important file called package.json. It stores information about the package using the JSON syntax and though it supports several standard keys/properties, only 2 of them are required: "name" and "version".

One can create and fill this file manually but fortunately, NPM comes with a built-in initializer for this file that helps a lot. To bring it up, one has to use following CLI command:

npm init

When this command is invoked, a questionnaire gets prompted for you to help the initializer fill the document. The questions are shown on your CLI one by one and they look like this:

package name: (react-from-scratch)
version: 0.0.1
description: Very cool react app built from scratch
entry point: src/app.tsx
test command:
git repository: <url-to-git-repo>
keywords: react typescript sass
author: Carlos Jasso
license: MIT

and the resulting file contains this:

{
  "name": "react-from-scratch",
  "version": "1.0.0",
  "description": "Very cool react app built from scratch",
  "main": "src/app.tsx",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "react",
    "typescript",
    "sass"
  ],
  "author": "Carlos Jasso",
  "license": "MIT"
}

🔬 Anatomy of the package.json file

Have you noticed that some values are given within parenthesis? Those are the default values that the initializer will write to the package.json if you just hit the ENTER key when the question prompts for the answer. You can change those values to anything of your preference.

Each of the fields represents the following:

• "name": contains your package's name. It must be written in lowercases, not include spaces but it may contain hyphens or underscores. Its default value is the directory's name.
• "version": this value must be in the <MajorRelease>.<MinorRelease>.<PatchRelease> form or x.x.x. Its default value is 1.0.0.
• "description": self explanatory, right? its value can even be an empty string "" (default).
• "main": it references the file that the assembly starts with. Commonly, one can find this field with the values "index.js", "app.js", "my-app-name.js". If you're not sure what to fill this with, you may as well leave it blank and will cause no harm.
• "scripts": its value is a nested json object with key/value entries for each of the "actions" needed to execute the package (or app) in one or other way. It by default has a "test" script that simply prints Error: no test specified to your CLI when executed.
• "keywords": an array of words that can be used to describe the package. It's common to find the names of the technologies involved in the project and by default, it's an empty value.
• "author": so that it's easy to find out whom a package was created by, it's convenient to add the author's information. It's suggested to follow the format <Name> <email@example.com> <http://example.com)> but just adding your name should be enough. An empty value is included by default.
• "license": a license should be specified to let people know how they are allowed to use your package and what constraints you've set for it. The value must be a SPDX license identifier which default value is "ISC".

🤓 Taking a shortcut

At this far, you may be wondering if there's a quicker way to create the package.json file. Fortunately, there's a way to indicate the NPM initializer to answer all the questions of the questionnaire with their default values. This is done by passing the -y argument to the command like this:

npm init -y

but this may not be as fun as entering your own data.

🧩 Dependency version specification

Most of the times, NODE apps are not only composed by the source code written by a developer but they also feature several packages written by others. In order to keep every dependency neat and tidy, whether they get used on the final product or only during build-time, the package.json file features a section where those dependencies are specified:

{
  ...,
  "dependencies": {
    "react": "^17.0.1",
    "react-dom": "~17.0.0"
  },
  "devDependencies": {
    "ts-loader": "8.0.0",
    "typescript": ">=4.1.0",
    "webpack": "<=5.0.0",
    "webpack-cli": "~4.5.0"
  }
}

Commonly, the name and version of each dependency is included in the package.json file and NPM will know what package to look for. Also, the path to a tarball or git URL can be specified.

The version value of a dependency is written in a rather specific notation that is pretty easy to understand:

• version: (no special characters) must match version exactly.
• >version: must be greater than version.
• >=version: must be greater than or equal to version.
• <version: must be lower than version.
• <=version: must be lower than or equal to version.
• ~version: approximately equivalent to version. Allow patch-level or minor-level changes.
• ^version: allow upgrades compatible with version.
• 1.2.x: could be 1.2.0, 1.2.1, etc., but not 1.3.0.
• *: matches any version
• "": (empty string) same as *
• version1 - version2: same as >=version1 <=version2.
• range1 || range2: passes if either range1 or range2 are satisfied.

At first, you may find that the package.json file doesn't contain neither a "dependencies" nor a "devDependencies" section, and that's because those sections aren't included when the file gets first initialized, however, it's important to know what these two sections are used for.

I hope you've found this article to be useful. Now you know how to properly initialize an NPM package and understand the contents of a package.json file a bit better.

If you'd like to go deeper into the topic, you may want to give this awesome document a read.

fun fact: this is my very first published article ever.

_{^{Cover Credits: Mockup psd created by freepik - www.freepik.com | Boboss74, CC BY-SA 4.0, via Wikimedia Commons}}