Transport Layer – TCP vs UDP, Three-Way Handshake, Flow Control and Congestion Control Explained



Transport Layer – TCP vs UDP, Three-Way Handshake, Flow Control and Congestion Control Explained

What You Will Learn

  • What the transport layer does and why it’s essential
  • TCP vs UDP — when to use each and why
  • TCP segment structure and important header fields
  • Three-way handshake and four-way connection termination
  • TCP flow control using the receive window (rwnd)
  • TCP congestion control: Slow Start, AIMD, Fast Retransmit
  • Socket programming basics and port numbers

1. Transport Layer Overview

The transport layer (Layer 4 in OSI) provides end-to-end communication between applications running on different hosts. While the Network layer (IP) gets packets from one computer to another, the Transport layer gets data from one specific application to another specific application.

Imagine you’re sending a large document across the internet. IP gets each packet from your computer to the destination computer. But TCP (at the transport layer) is what breaks the document into packets, numbers them so they can be reassembled in order, confirms each one arrives, and retransmits any that get lost.

Transport Layer Services

  • Multiplexing/Demultiplexing: Multiple applications on one host can use the network simultaneously — distinguished by port numbers
  • Reliable delivery (TCP only): Acknowledgements, retransmission, ordering
  • Flow control (TCP only): Prevents sender from overwhelming receiver
  • Congestion control (TCP only): Prevents sender from overwhelming the network
  • Error checking: Both TCP and UDP use checksums to detect corrupted segments

Ports and Multiplexing

A port number (16-bit integer, 0–65535) identifies which application should receive data on a host. The combination of (IP address, port number) is called a socket. A TCP connection is uniquely identified by the 4-tuple: (source IP, source port, destination IP, destination port).

2. TCP vs UDP

FeatureTCPUDP
ConnectionConnection-oriented (handshake required)Connectionless (no setup)
ReliabilityGuaranteed delivery with ACKs and retransmissionBest-effort, no guarantee
OrderGuarantees in-order deliveryNo ordering guarantee
Flow controlYes (receive window)No
Congestion controlYes (slow start, AIMD)No
Error checkingYes (checksum)Yes (checksum, optional in IPv4)
Header size20–60 bytes (variable)8 bytes (fixed)
SpeedSlower (overhead of reliability)Faster (minimal overhead)
Use casesHTTP/S, FTP, SMTP, SSH, database queriesDNS, DHCP, VoIP, video streaming, gaming
Why use UDP? For real-time applications (video calls, live streaming, online games), a slightly delayed retransmission is worse than no retransmission — you’d rather have a glitch than a freeze. UDP lets the application decide how to handle loss. Also, DNS uses UDP for its tiny request/response messages — reliability overhead would be wasteful for 40-byte queries that time out and retry anyway.

3. TCP Segment Structure

FieldSizePurpose
Source Port16 bitsSender’s port number
Destination Port16 bitsReceiver’s port number
Sequence Number32 bitsPosition of first data byte in this segment within the stream
Acknowledgement Number32 bitsNext sequence number the receiver expects (cumulative ACK)
Header Length4 bitsSize of TCP header in 32-bit words (minimum 5, i.e., 20 bytes)
Flags9 bitsSYN, ACK, FIN, RST, PSH, URG, ECE, CWR, NS
Receive Window (rwnd)16 bitsBuffer space available at receiver — used for flow control
Checksum16 bitsError detection over header + data
Urgent Pointer16 bitsOffset to urgent data (when URG flag set)
Options0–40 bytesMSS negotiation, SACK, timestamps, window scaling
DataVariableApplication payload
Key TCP flags:
SYN — synchronise sequence numbers (connection setup)
ACK — acknowledgement field is valid
FIN — no more data from sender (connection teardown)
RST — reset connection immediately (error condition)
PSH — push data to application immediately (don’t buffer)
URG — urgent data present

4. Three-Way Handshake

TCP uses a three-way handshake to establish a reliable connection and synchronise sequence numbers before any data is sent.

Step 1 — SYN (Client → Server):
Client sends: SYN flag set, Seq = x (client’s random ISN)
“I want to connect. My starting sequence number is x.”

Step 2 — SYN-ACK (Server → Client):
Server sends: SYN + ACK flags, Seq = y (server’s ISN), Ack = x+1
“I agree. My starting sequence number is y. I’ve received your SYN (x), next I expect x+1.”

Step 3 — ACK (Client → Server):
Client sends: ACK flag, Seq = x+1, Ack = y+1
“Got it. Next I expect y+1 from you.”

Connection established. Data transfer begins.

Why three steps? Both sides need to announce their Initial Sequence Numbers (ISNs) and confirm the other side’s ISN. Two messages are enough for one side, but each side needs its own SYN acknowledged — hence three messages minimum. (The server’s SYN and ACK are combined into one message.)

SYN Flood Attack: An attacker sends many SYN requests without completing the handshake. The server allocates resources for each half-open connection, eventually running out. Defence: SYN cookies — the server doesn’t allocate state until the final ACK is received, encoding state in the sequence number instead.

5. Connection Termination (Four-Way Handshake)

TCP connection termination is independent in each direction — each side closes its half of the connection separately.

Step 1 — FIN (Active closer → Passive):
“I have no more data to send.” (But can still receive)

Step 2 — ACK (Passive → Active):
“Got your FIN. I’ll close my side when ready.”

Step 3 — FIN (Passive → Active):
“I also have no more data to send now.”

Step 4 — ACK (Active → Passive):
“Got your FIN.” → Active closer enters TIME_WAIT state (2×MSL, typically 2 minutes)

Connection fully closed.

TIME_WAIT state: After sending the final ACK, the active closer waits 2×MSL (Maximum Segment Lifetime, typically 2 minutes) before fully closing. This ensures the final ACK (which might be lost) can be retransmitted if needed, and ensures any delayed packets from the old connection don’t interfere with new connections using the same port.

6. TCP Flow Control

Flow control prevents the sender from transmitting faster than the receiver can process and buffer data. The receiver advertises its receive window (rwnd) — available buffer space — in every ACK.

Sender’s constraint:
Amount of unacknowledged data in flight ≤ rwnd

If rwnd = 0: sender stops transmitting
Sender periodically sends 1-byte probe segments to check if rwnd has opened

Effective throughput ceiling:
Maximum throughput ≤ rwnd / RTT

Example:
Receiver’s buffer = 65,535 bytes (max with 16-bit rwnd field)
RTT = 100 ms
Max throughput = 65,535 bytes / 0.1 s ≈ 5.24 Mbps

For Gigabit networks, the 16-bit rwnd is a bottleneck! TCP Window Scaling Option (in Options field) multiplies rwnd by 2^shift factor (up to 2^14) to support large receive windows on high-bandwidth-delay links.

7. TCP Congestion Control

Congestion control prevents TCP from overwhelming the network (routers and links along the path). TCP infers network congestion from packet loss events and adjusts its sending rate accordingly.

Congestion Window (cwnd)

TCP maintains a congestion window (cwnd) — the maximum amount of data that can be “in flight” (sent but not yet acknowledged) at any time. The effective send rate is governed by: min(cwnd, rwnd).

Slow Start

  • New connections start with cwnd = 1 MSS (Maximum Segment Size, typically 1460 bytes)
  • For each ACK received, cwnd increases by 1 MSS → effectively doubling cwnd each RTT
  • Exponential growth until cwnd reaches ssthresh (slow start threshold)
  • Despite the name, slow start grows exponentially — it’s “slow” only relative to the instantaneous maximum

Congestion Avoidance

  • Once cwnd ≥ ssthresh, switch to linear increase
  • cwnd increases by 1 MSS per RTT (additive increase)
  • Continues until congestion is detected

On Packet Loss — TCP Reno Behaviour

Loss EventInferenceAction
Timeout (no ACK received)Severe congestionssthresh = cwnd/2; cwnd = 1 MSS; restart slow start
3 duplicate ACKsMild congestion (packet lost but others getting through)ssthresh = cwnd/2; cwnd = ssthresh; enter congestion avoidance (TCP Reno fast recovery)
AIMD — Additive Increase Multiplicative Decrease:
Increase: cwnd += 1 MSS per RTT (additive)
Decrease: cwnd = cwnd/2 on loss (multiplicative)

This gives TCP its characteristic “sawtooth” sending rate pattern over time.

TCP throughput (simplified):
Throughput ≈ (0.75 × W) / RTT   (where W = max window size when loss occurs)

More accurate: Throughput ≈ (1.22 × MSS) / (RTT × √p)
where p = packet loss probability

Congestion control trace:
ssthresh = 8, MSS = 1
Round 1: cwnd=1 (slow start)
Round 2: cwnd=2
Round 3: cwnd=4
Round 4: cwnd=8 (hit ssthresh → switch to congestion avoidance)
Round 5: cwnd=9
Round 6: cwnd=10
Round 7: cwnd=11 → timeout! ssthresh=5 (11/2 rounded), cwnd=1
Round 8: cwnd=1 (slow start again)
Round 9: cwnd=2
Round 10: cwnd=4
Round 11: cwnd=5 (hit new ssthresh → congestion avoidance)

8. UDP in Detail

UDP’s simplicity is its strength. The entire UDP header is just 8 bytes.

FieldSizePurpose
Source Port16 bitsOptional sender port (0 if unused)
Destination Port16 bitsReceiver’s application port
Length16 bitsTotal length of UDP header + data
Checksum16 bitsError detection (optional in IPv4, mandatory in IPv6)

When UDP is the Right Choice

  • Real-time applications: VoIP, video conferencing — stale retransmitted data is useless
  • Short transactions: DNS — tiny request/response, simpler to retry than maintain TCP state
  • Broadcast/Multicast: TCP is point-to-point only; UDP supports one-to-many
  • Streaming: Netflix-style video uses TCP, but live streams often use UDP or QUIC
  • Gaming: Online games prefer low latency over reliability; game logic handles loss
QUIC: Google developed QUIC (now standardised as HTTP/3) which runs over UDP but implements its own reliability, multiplexing, and encryption — getting the benefits of both TCP reliability and UDP speed while eliminating TCP’s head-of-line blocking and handshake latency.

9. Port Numbers

ProtocolPortTransportPurpose
FTP (data)20TCPFile transfer — data channel
FTP (control)21TCPFile transfer — commands
SSH22TCPSecure remote shell
Telnet23TCPUnsecured remote shell (legacy)
SMTP25TCPSending email (server-to-server)
DNS53UDP (+ TCP for large)Domain name resolution
DHCP (server)67UDPDHCP server listens here
DHCP (client)68UDPDHCP client listens here
HTTP80TCPWeb (unencrypted)
POP3110TCPReceiving email (download)
IMAP143TCPReceiving email (sync)
HTTPS443TCPWeb (encrypted with TLS)
RDP3389TCPRemote Desktop Protocol

Port ranges:

  • 0–1023: Well-known/system ports — reserved for standard services, require root/admin to bind
  • 1024–49151: Registered ports — assigned to specific applications by IANA
  • 49152–65535: Ephemeral/dynamic ports — used by OS for client-side of connections (source port)

10. Common Misconceptions

  • “TCP is always better than UDP”: TCP’s reliability features add latency and overhead. For real-time applications (VoIP, live video, gaming), UDP is superior because retransmissions arrive too late to be useful. Choosing between TCP and UDP depends entirely on the application’s requirements.
  • “Slow start is slow”: Slow start doubles the congestion window every RTT — that’s exponential growth. It’s “slow” only in that it doesn’t immediately blast data at full speed, avoiding immediate congestion. In practice, slow start can reach network capacity in just a few RTTs.
  • “Flow control and congestion control are the same”: Flow control is between sender and receiver — it prevents the receiver’s buffer from overflowing. Congestion control is between sender and the network — it prevents router queues from overflowing. Both use the concept of a window, but they’re addressing different bottlenecks.
  • “ACK numbers in TCP are cumulative”: This is actually correct — but students sometimes think each ACK acknowledges exactly one segment. TCP’s ACK number says “I’ve received everything up to byte X-1; send me X next.” A single ACK can acknowledge multiple segments (delayed ACKs).
  • “UDP has no error checking”: UDP includes a checksum field. What it lacks is error recovery — it detects errors but simply discards the corrupted datagram. The application must handle retransmission if needed.

11. Frequently Asked Questions

What is the difference between TCP and UDP?

TCP is connection-oriented and reliable: three-way handshake, ordered delivery, ACKs, retransmission, flow and congestion control. 20-byte minimum header. Used for HTTP, FTP, SMTP. UDP is connectionless and unreliable: no handshake, no ordering, no ACKs, no congestion control. 8-byte fixed header. Used for DNS, VoIP, streaming, gaming. Choose TCP when data integrity matters; choose UDP when speed and low latency matter more than perfect delivery.

How does TCP three-way handshake work?

Step 1: Client sends SYN with its Initial Sequence Number (ISN). Step 2: Server responds with SYN+ACK — acknowledges client’s ISN (ACK = client_ISN + 1) and announces its own ISN. Step 3: Client sends ACK acknowledging server’s ISN (ACK = server_ISN + 1). Both sides have now exchanged and acknowledged sequence numbers. Data transfer begins. Three steps are the minimum needed for both sides to announce and confirm their sequence numbers bidirectionally.

What is TCP congestion control and how does it work?

TCP infers network congestion from packet loss. In Slow Start, cwnd doubles each RTT until ssthresh is reached. In Congestion Avoidance, cwnd grows by 1 MSS per RTT (linear). On timeout (severe loss): ssthresh = cwnd/2, cwnd = 1, restart Slow Start. On 3 duplicate ACKs (TCP Reno): ssthresh = cwnd/2, cwnd = ssthresh, continue Congestion Avoidance (fast recovery). This AIMD pattern — additive increase, multiplicative decrease — gives TCP its sawtooth sending rate and is what makes TCP fair among competing flows.

What is the difference between flow control and congestion control in TCP?

Flow control: receiver-driven, prevents its own buffer overflow; receiver advertises rwnd (receive window) in ACKs; sender keeps unacknowledged data ≤ rwnd. Congestion control: network-driven, prevents router/link overload; sender tracks cwnd (congestion window) based on loss events; sender keeps unacknowledged data ≤ min(cwnd, rwnd). Both limit the sender’s rate but for different reasons — flow control protects the receiver, congestion control protects the network.

What port numbers are used by common TCP/UDP protocols?

Key ports: HTTP=80, HTTPS=443, FTP=20/21, SSH=22, Telnet=23, SMTP=25, DNS=53 (UDP+TCP), DHCP=67/68 (UDP), POP3=110, IMAP=143. Ports 0–1023 are well-known system ports requiring admin privileges. Ports 1024–49151 are registered. Ports 49152–65535 are ephemeral (used by OS for client connection source ports). DNS is unusual in using both UDP (small queries) and TCP (large responses or zone transfers).