# ESP32 FTP_Server

## Overview
- File Transfer Protocol (FTP), short for File Transfer Protocol.
FTP is a client/server system. The user connects to the FTP server program on the remote host through a client program that supports the FTP protocol. 
The user issues a command to the server program through the client program. The server program executes the command issued by the user and returns the result of the execution to the client.
When the FTP client establishes an FTP connection with the server, it will establish contact with the two ports on the server: ports 20 and 21 (not fixed).
FTP uses different port numbers to transmit different content and establishes different TCP connections. First, use TCP to generate a virtual connection for control information, 
and then generate a separate TCP connection for data transfer.
- Design ESP32 as FTP_Server:
  - First need to set ESP32 to **SoftAP** mode;
  - Connect SDcard to ESP32, initialize SDcard, and use SDcard as file storage media of FTP_Server;
  - Networking at the client (PC) so that the PC can be connected to ESP32 via WIFI;
  - Log on at the client (PC) using the FTP client software (**WinSCP** is used in this article). The IP address of the login host is ESP32. The port number is the default port 21. 
	The username and password are set to: *esp32*
    *Note: The host name, user name and password can be modified in the program. *

## Installation Instructions for Hardware and Software Environment
### Hardware:

|Name |Description |Remarks |
|:------------:|:------:|:----------:|
|ESP-WROVER-KIT|Server |
|SDcard |Storage Device | |
|PC |Client |Wireless Network Card|

### Software:

 - WinSCP software:
WinSCP is an open source free SFTP client for Windows, FTP client, WebDAV client and SCP client. Its main function is to transfer files between local and remote computers. 
In addition, WinSCP provides scripting and basic file manager functionality.
WinSCP includes a graphical user interface, multiple languages, integration with Windows, batch file scripting and command line interface integration, and a variety of other useful features.
 - code key settings in the program
  - IP address:
          IP4_ADDR(&esp_ip[TCPIP_ADAPTER_IF_AP].ip, 192, 168, 4, 1);
          IP4_ADDR(&esp_ip[TCPIP_ADAPTER_IF_AP].gw, 192, 168, 4, 1);
          IP4_ADDR(&esp_ip[TCPIP_ADAPTER_IF_AP].netmask, 255, 255, 255, 0);
  - Username and password settings:
  We use two macro definition implementations:
          #define FTP_USER "esp32"
          #define FTP_PASS "esp32"
  - Passive mode settings
          Else if (!strcmp(command, "PASV")) {
            If (dataStatus != NODATACONN)
                dataConnClose();
            sendCmdConn("227 Entering Passive Mode (192,168,4,1,5,20).");
            dataConnCreate();
            Return true;
          }
  After the server receives the PASV command from the client, it uses the command port (port 21) to return the server's IP address and open data transmission port.
         IP Address: 192.168.4.1
         Data port: 5*256 + 20 = 1300
  - Command port and data port settings:
         #define FTP_CTRL_PORT 21 // Command port on wich server is listening
         #define FTP_DATA_PORT_PASV 1300 // Data port for passive mode

## Software Design Description
### FTP Protocol Connection Overview
FTP shields the details of each computer system and is therefore suitable for transferring files between any computer in a heterogeneous network. FTP only provides some basic services for file transfer. 
It uses TCP to reliably transport services. The main function of FTP is to reduce or eliminate the incompatibility of processing files under different systems.
FTP uses a client-server model where an FTP server process can serve multiple client processes. 
The FTP server consists of two major components: a main process that is responsible for accepting new requests, and a number of dependent processes that handle a single request. 
A double connection is established between the FTP client and the server. One is a control connection and the other is a data connection.
![](http://i.imgur.com/ZF5EGeU.png)

FTP uses 2 TCP ports, a data port, and a command port (also called a control port).
When a command connection is established, port 21 is generally used as a command port, and the client connects to an FTP server's command port, 
ie port 21, from an arbitrary unprivileged port N (N?1024) to establish a control connection. 
This connection is used to pass the client's command and the server's response to the command. The lifetime is the entire FTP session time.
  
 When establishing a data connection, the client needs to establish a data connection if it needs to transfer files and other data during the period,such as a directory list. 
 This connection is established when data transmission is required, and once the data is transmitted, it is closed and may be established multiple times during the entire FTP. 
 In active mode, when a data connection is established, the client will start listening on port N+1 and send the FTP command port N+1 to the FTP server. 
 The server then connects from its own data port (20) to the client-specified data port (N+1) to begin data transmission.
 * When mixed with the concept of active/passive mode, the data port may not be 20.*

### Active Mode FTP:
In active mode, the FTP client is connected to the FTP server's command port - 21 from any non-special port (N > 1023). 
The client then listens on the N+1 (N+1 >= 1024) port and sends the command to the FTP server over the N+1 (N+1 >= 1024) port. 
The server will in turn connect to the user's locally specified data port, such as 20 ports.
With a server-side firewall as a starting point, to support active-mode FTP, open the ports used in the following interactions:

     1. FTP server command (21) port accepts any client port (client initial connection)
     2. FTP server commands (21) port to client port (>1023) (server responds to client commands)
     3. FTP server data (20) port to client port (>1023) (server initialization data is connected to client data port)
     4. The FTP server data (20) port accepts the client port (>1023) (the client sends an ACK packet to the server's data port)

The diagram is as follows:
  
  ![](http://i.imgur.com/xLR4LRL.jpg)
  
    In step 1, the client's command port establishes a connection with the FTP server's command port and sends the command "PORT 1027";
    In step 2, the FTP server returns an "ACK" to the client's command port;
    In step 3, the FTP server initiates a connection from its own data port (20) to the client's previously specified data port (1027);
    In step 4, the final client returns an "ACK" to the server.
        
 **The main problems with Active FTP: **
 For the client's firewall, this is the connection from the external system to the internal client, which is usually blocked.
  
### Passive Mode FTP:
  
 ** The software design in this example uses passive mode.**
Passive mode first requires the client to tell the server that the client is in passive mode, and the server will enable the corresponding passive mode (PASV).
  
 In passive FTP, both the command connection and the data connection are from the client. 
 This can solve the problem that the incoming connection of the data port from the server to the client is filtered out by the firewall. 
 When opening an FTP connection, the client opens two arbitrary unprivileged local ports (N > 1024 and N + 1). The first port connects to server's port 21, 
 but unlike active mode FTP, the client does not submit the PORT command and allows the server to connect back and forth to its data port. 
 Instead, it submits the PASV command. The result of this is that the server will open an arbitrary unprivileged port (P > 1024) and send PORT P and the server's IP address to the client. 
 The client then initiates a connection from the local port N+1 to the server's port P for data transfer.
The diagram is as follows:

 ![](http://i.imgur.com/Uhk2Wz3.jpg)
   
    In step 1, the client's command port establishes a connection with the server's command port and sends the command "PASV";
    In step 2, the server returns port 2024 and server IP, telling the client which port to use to listen for data connections;
    In step 3, the client initiates a data connection from its own data port to the server-specified data port;
    In step 4, an "ACK" response is returned to the client's data port.

## Software Design Description
During the connection and data transmission process, the client will send a variety of commands to the server, the server will first parse the command, 
and then according to the different commands to deal with accordingly.
** The main FTP commands used are: **

|Order |Description |Remarks |
|:-----:|:-----------------:|:-------------------- -------------------------------------------------- -----|
|USER |Specify the username |usually the first command issued after controlling the connection. "USER esp32\r\n": username is esp32 login |
|PASS |Specify User Password |This command immediately follows the USER command. "PASS esp32\r\n": The password is esp32 |
|SIZE|returns the size of the specified file |"SIZE file.txt\r\n": returns the file size if the file.txt file exists |
|CWD |Change Working Directory |Example:"CWD dirname\r\n" |
|PASV |Enter Passive Mode |Example:"PASV\r\n" |
|RETR |Download file |"RETR file.txt \r\n": Download file file.txt |
|PWD |print the current directory |eg:"PWD\r\n" |
|LIST |Listing directories |"LIST -a\r\n": List all directories |

**User Response Code: **
  
After the client sends an FTP command to the server, the server returns a three-digit coded response code.

    The first number gives a general indication of the status of the command, such as the success, failure, or incompleteness of the response.
    The second number is the classification of the response type. For example, 2 represents the connection-related response, and 3 represents the user authentication.
    The third number provides more detailed information.
  
E.g:   

    331 User esp32 accepted, provide password.
    230 User esp32 logged in.
    257 \"sdcard\" is your current location
  
**LIST Return data format: **

 - MS-DOS file list format resolution
        02-23-05 09:24AM 33 readme.md
        05-25-04 08:56AM 33 VC.c
 - UNIX file list format resolution
        -rwxrw-r-- 1 user user 3014 Nov 12 14:57 readme.md
        -rwxrw-r-- 1 user user 20480 Mar 3 11:25 VC.c
 - Windows comes with FTP:
        -rwxrwxrwx 1 owner group 19041660 May 25 2004 VC.ESn
 
In this routine, the UNIX file format is used and set in the SYST command return function in the program.

        Else if (!strcmp(command, "SYST")) {
            sendCmdConn("214 UNIX system type.");
        }

## Test Example
We use the client (PC) to use Wireshark to capture packets, print with the serial port of the server, and verify the experiment.
### log in  
Enable ESP32 in SoftAP mode. The client (PC) starts WinSCP, sets up the site, and logs in:
 - Select the file protocol as FTP, encryption method choose not to encrypt;
 - Host name server IP address: 192.168.4.1;
 - The port number is the command transmission port: 21;
 - The username and password are *esp32* set in the program

![](http://i.imgur.com/uUgWd54.png)

**Click to login, corresponding to packet capture analysis:**
  
![](http://i.imgur.com/iSrPvYx.png)

1. First, three TCP handshakes between the ports 52281 and 21 for the server and client establish a command transmission connection. The IP address of the client is: 192.168.4.2
  
  ![](http://i.imgur.com/co8Cim6.png)

2. Immediately after the server sends a welcome message to the client through the command port:

        220 -- Welcom connect to ESP32 FTP
    
3. The client sends USER and PASS commands to verify the login name and password and obtain the server authentication response.


**Serial information printing on the server-side ESP32:**

    -------------------------------------------------- -------
    Query: USER esp32
    Command: 'USER'|Parameter: 'esp32'
    The user is esp32
    Response: 331 User esp32 accepted, provide password.
    -------------------------------------------------- -------
    Query: PASS esp32
    Command: 'PASS'| Parameter: 'esp32'
    Response: 230 User esp32 logged in.
    -------------------------------------------------- -------

### Server Properties Get

**Capture analysis:**

![](http://i.imgur.com/lRTqkfS.png)

Here we can see that the server's system is set to UNIX system and the command FEAT is recognized.

**View Serial Print:**
  
-------------------------------------------------- -------
Query: SYST
    Command: 'SYST'|Parameter: ''
    Response: 214 UNIX system type.
The
-------------------------------------------------- -------
Query: FEAT
    Command: 'FEAT'| Parameter: ''
    Response: 502 FEAT Command not implemented.
The
-------------------------------------------------- -------

### Print FTP Current Directory

We set the directory of the SDcard as the default home directory of the login, so we first get the directory of the SDcard.

**Capture analysis:**

![](http://i.imgur.com/EDGZPk4.png)

This process can be divided into 4 steps:
  
Step 1: Process the PWD and CWD commands. After the server receives the PWD command, it will return the current directory name, ie **sdcard**, on the server. 
After the client receives the return, the directory name will be extracted and sent. 
The CWD command changes the directory to **/sdcard** and then sends the print directory command again after changing the directory.

The second step: Get the server data transmission type, get the server back, the data transmission type is ASCII type;

The third step: set to enter the PASV passive mode, in the change mode, start the data transmission, the server returns the current IP address and open data port number: 1300, 
and sends a LIST command to request list content;

Step 4: After the client obtains the server's address and port, it initiates a TCP handshake on the three sides and establishes a data connection.

Step 5: The server uses the data connection to send the current directory information to the client.

- Open the directory return information packet:

![](http://i.imgur.com/LFrHwqE.png)

The data indicates that the data format is a UNIX file type, and there is only one file FOO.TXT in the current directory, and the file size is 1637 bytes.

**View Serial Print:**

-------------------------------------------------- -------
Query: CWD /sdcard
Command: 'CWD '|Parameter: '/sdcard'
Response: 250 Directory is changed
-------------------------------------------------- -------
Query: PWD
Command: 'PWD '|Parameter: ''
Response: 257 "sdcard" is your current location
-------------------------------------------------- -------
Query: TYPE A
Command: 'TYPE'|Parameter: 'A'
Response: 200 TYPE is now ASCII
-------------------------------------------------- -------
Query: PASV
Command: 'PASV'|Parameter: ''
Response: 227 Entering Passive Mode (192,168,4,1,5,20).
    -------------------------------------------------- -------
    Query: LIST -a
Command: 'LIST'|Parameter: '-a'
Ready to list
In DataConnConnect!
Response: 150 Accepted data connection
File'name is FOO.TXT
File end
In dataConnClose. Closing down data connection!
Response: 226 1 matches total
-------------------------------------------------- -------
    
The client shows the following:

![](http://i.imgur.com/s20C8y4.png)


### Download Document  

Right-click on the file in the FTP directory to download and enter the download settings page:

![](http://i.imgur.com/k0TaMKq.png)

**Capture analysis:**

![](http://i.imgur.com/wN76q8h.png)

First, the client sends a RETR command to the server, and indicates that the download object is FOO.TXT. A new TCP TCP handshake takes place. 
After the connection is established, the new data connection is used to send the file FOO.TXT to the client. The client finishes receiving.

**View Serial Print:**

    -------------------------------------------------- -------
Query: PASV
Command: 'PASV'|Parameter: ''
Response: 227 Entering Passive Mode (192,168,4,1,5,20).
-------------------------------------------------- -------
Query: RETR FOO.TXT
Command: 'RETR'|Parameter: 'FOO.TXT'
File = /sdcard/FOO.TXT
In DataConnConnect!
Response: 150 Accepted data connection
In Retrieve Loop!
In dataConnClose. Closing down data connection!
In dataConnRecvCb
    -------------------------------------------------- -------

At this point, a complete FTP server file read and download transfer is complete.