Skip to content

Commit b91ab8d

Browse files
FayisRahmanFayisRahman
committed
added stage 18 doc
1 parent c9e17a6 commit b91ab8d

1 file changed

Lines changed: 200 additions & 5 deletions

File tree

‎docs/roadmap/phase-3/stage-18.md‎

Lines changed: 200 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,215 @@
22

33
## Recap
44

5-
In Phase 2, we have made `expServer` HTTP compatible and dynamically configurable using JSON file. We have also implemented directory browsing as a fallback when no index file is present.
5+
In Phase 2, we have made expServer HTTP compatible and dynamically configurable using JSON file. We have also implemented directory browsing as a fallback when no index file is present.
66

77
## Learning Objectives
88

99
- We will implement IP whitelist/blacklist functionality
1010

11-
<!-- :::tip PRE-REQUISITE READING
11+
:::tip PRE-REQUISITE READING
1212

13-
Read about IP whitelist/blacklist functionality from [here](https://instasafe.com/blog/whitelisting-vs-blacklisting-whats-the-difference/)
13+
Read about [IP whitelist/blacklist](https://instasafe.com/blog/whitelisting-vs-blacklisting-whats-the-difference/)
1414

15-
::: -->
15+
:::
1616

1717
## Introduction
1818

19-
IP whitelisting and blacklisting are security features that allow you to control access to your web server based on IP addresses. IP whitelisting allows only specified IP addresses to access your server, while IP blacklisting blocks specified IP addresses from accessing your server.
19+
Security in web servers isn't just about encryption and passwords; it's also about controlling *who* can reach your server in the first place. **IP Whitelisting and Blacklisting** are foundational access control mechanisms that allow you to filter traffic based on the client's origin, either by creating a "trusted" circle of allowed addresses or by proactively shielding against known malicious actors, bots, and spam sources.
2020

2121
## Implementation
22+
23+
In this stage, we will be adding IP-based access control to the server, allowing or denying requests based on the client's IP address. This enhances security by restricting access to specific IPs.
24+
25+
### xps_config
26+
27+
#### `xps_config.h`
28+
We extend the configuration structures to include IP lists.
29+
30+
- Add to `xps_config_route_s`:
31+
- `ip_whitelist (vec_void_t)`
32+
- `ip_blacklist (vec_void_t)`
33+
- Add the same fields to `xps_config_lookup_s` as well
34+
35+
#### `xps_config.c`
36+
Updated parsing logic to handle IP lists in the JSON configuration.
37+
38+
- In `parse_server()`: Initialize the `ip_whitelist` and `ip_blacklist` vectors of the `route` structure with `vec_init(&route->ip_whitelist)` and `vec_init(&route->ip_blacklist)`.
39+
40+
- The configuration handles IP lists with the following precedence:
41+
42+
- **Whitelist only**: Only the specified IP addresses are granted access.
43+
- **Blacklist only**: All IP addresses are allowed except those specifically listed.
44+
- **Both present**: The whitelist takes priority. We will filter the `ip_whitelist` by removing any IPs that also appear in the `ip_blacklist`, ensuring only trusted, non-blocked addresses remain.
45+
46+
:::details expserver/src/config/xps_config.c `parse_route()`
47+
48+
```c
49+
void parse_route(JSON_Object *route_object, xps_config_route_t *route) {
50+
51+
...
52+
53+
JSON_Array *ip_whitelist = json_object_get_array(route_object, "ip_whitelist");
54+
JSON_Array *ip_blacklist = /*fill this*/
55+
56+
if(ip_whitelist && !ip_blacklist){
57+
for(size_t i = 0; i < /*iterate through ip_whitelist array*/; i++)
58+
vec_push(&(route->ip_whitelist), /*get ith IP from ip_whitelist array*/);
59+
}else if(ip_blacklist && !ip_whitelist){
60+
/*fill this*/
61+
}else if(ip_whitelist && ip_blacklist) {
62+
/*fill this*/
63+
}
64+
}
65+
```
66+
67+
:::
68+
69+
70+
71+
- In `xps_config_lookup()`: Copy the `ip_whitelist` and `ip_blacklist` pointers from the matched route into the lookup structure <br> (ie, by doing `lookup->ip_whitelist = route->ip_whitelist` etc..)
72+
73+
74+
### xps_session
75+
76+
#### `xps_session.c`
77+
In `session_process_request()`, we add IP filtering logic before processing the request.
78+
79+
1. Check if the client IP is in the whitelist (if **whitelist** exists).
80+
2. Check if the client IP is in the blacklist (if b**lacklist** exists).
81+
3. If access is denied, return `HTTP_FORBIDDEN (403)` status.
82+
83+
:::details expserver/src/core/xps_session.c `session_process_request()`
84+
85+
```c
86+
void session_process_request(xps_session_t *session) {
87+
88+
...
89+
90+
session->lookup = lookup;
91+
92+
// check whitelist exist
93+
if (lookup->ip_whitelist.length > 0) {
94+
const char *client_ip = session->client->remote_ip;
95+
bool is_allowed = false;
96+
for (size_t i = 0; i < lookup->ip_whitelist.length; i++) {
97+
const char *ip_w = lookup->ip_whitelist.data[i];
98+
if (strcmp(client_ip, ip_w) == 0) {
99+
is_allowed = true;
100+
break;
101+
}
102+
}
103+
if (!is_allowed) {
104+
logger(LOG_DEBUG, "session_process_request()", "client ip %s is not whitelisted", client_ip);
105+
xps_http_res_t *http_res = /*create http response with status code HTTP_FORBIDDEN*/
106+
xps_buffer_t *http_res_buff = /*serialize the http response*/
107+
/*set the response to client buffer*/
108+
/*destroy the http response*/
109+
return;
110+
}
111+
}
112+
113+
// check in blacklist
114+
if (lookup->ip_blacklist.length > 0) {
115+
const char *client_ip = /*fill this*/
116+
for (size_t i = 0; i < /*fill this*/; i++) {
117+
const char *ip_b = /*fill this*/
118+
if (/*fill this*/) {
119+
logger(LOG_DEBUG, "session_process_request()", "client ip %s is blacklisted", client_ip);
120+
/*fill this*/
121+
return;
122+
}
123+
}
124+
}
125+
126+
...
127+
}
128+
```
129+
:::
130+
131+
With this we have implemented the whitelisting and blacklisting functionality to our expserver.
132+
133+
## Milestone
134+
135+
By completing this stage, you have successfully added a vital security layer to your server. Verify your implementation by following the steps below:
136+
137+
Add `ip_whitelist` or `ip_blacklist` fields to your `xps_config.json` for specific routes. For example:
138+
```json
139+
{
140+
"routes": [
141+
{
142+
"req_path": "/",
143+
"type": "file_serve",
144+
"ip_whitelist": ["127.0.0.1", "0.0.0.0"],
145+
"ip_blacklist": ["127.0.0.2"]
146+
}
147+
]
148+
}
149+
150+
```
151+
152+
You can test all three scenarios directly from your terminal:
153+
- **Blacklist**: Add `127.0.0.1` to the blacklist and confirm you receive a `403 Forbidden`.
154+
- **Whitelist (Blocked)**: Set the whitelist to a dummy IP (e.g., `1.1.1.1`) and confirm access is denied.
155+
- **Whitelist (Allowed)**: Add `127.0.0.1` back to the whitelist and confirm a `200 OK`.
156+
157+
You can test with multiple IPs without the use of any VPNs. The entire `127.x.x.x` range points back to your machine. You can simulate requests from "different" users by forcing `curl` to bind to an alias IP:
158+
```bash
159+
curl --interface 127.0.0.2 -i http://0.0.0.0:8001/sample.txt
160+
```
161+
The `--interface` flag is used to specify the source IP address for the request and the `-i` flag is used to print the HTTP headers.
162+
If you add `127.0.0.2` to your blacklist, this command will trigger the 403 block perfectly.
163+
164+
With these tests passed, you are ready to move on to the next phase of optimization!
165+
166+
## Experiments
167+
168+
### Experiment #1
169+
170+
171+
First, create a relatively large text file in your `public` directory. You can use the `yes` command to generate a 1MB file filled with repeated text:
172+
173+
```bash
174+
# Generate a 1MB file of repeated text
175+
yes "This is a long line of text for testing " | head -c 1048576 > public/large.txt
176+
```
177+
178+
To see exactly what the server thinks it's sending, let's add a temporary `printf` in `xps_session.c` within the `session_process_request` function, specifically where we handle file serving:
179+
180+
Inside `session_process_request()`
181+
```c
182+
183+
if (lookup->file_path) {
184+
...
185+
session->file = file;
186+
printf("[Experiment] Serving file: %s | Size: %zu bytes\n", lookup->file_path, session->file->size); // [!code ++]
187+
...
188+
}
189+
```
190+
191+
192+
Start your server and use `curl` to request the file. We'll use the `-I` flag to see the headers:
193+
194+
```bash
195+
curl -I http://localhost:8001/large.txt
196+
```
197+
198+
In your server console, you should see:
199+
```log
200+
[Experiment] Serving file: public/large.txt | Size: 1048576 bytes
201+
202+
```
203+
204+
In the `curl` output, look for the `Content-Length` header:
205+
`Content-Length: 1048576`
206+
207+
Currently, every single byte of the file is being sent over the wire, which is highly inefficient for larger files.
208+
209+
In the next stage, we will implement **Compression** to reduce this size and optimize data transfer. By compressing the response on the fly, we can minimize the amount of data sent over the wire, making our server faster and more bandwidth-efficient.
210+
211+
212+
## Conclusion
213+
214+
With the implementation of IP-based access control, you have added a foundational security layer to eXpServer. By allowing or denying requests based on the client's IP, you can now protect sensitive routes or block malicious actors. However, keep in mind that IP filtering is just one part of a "defense-in-depth" strategy; in production, you should always complement it with HTTPS and proper authentication to protect against spoofing or proxy-based bypasses.
215+
216+
Now that we have secured our routes, it's time to focus on performance. In the next stage, we will learn how to compress our HTTP responses on the fly. By using compression algorithms, we can reduce the size of the data being sent over the wire. This not only saves bandwidth but also improves the loading speed for your users, making your server feel much more responsive and professional.

0 commit comments

Comments
 (0)