1
0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2025-11-23 21:54:53 +02:00
Commit Graph

39 Commits

Author SHA1 Message Date
Michael Niedermayer
c199b3d48f avformat/whip: Fix rtp_ctx->streams access
Fixes: out of array access
No testcase

Found-by: Joshua Rogers <joshua@joshua.hu> with ZeroPath
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-11-10 00:24:44 +00:00
Jack Lau
cb5e201f5c avformat/whip: cleanup the redundant variable
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-23 10:58:45 +00:00
Jack Lau
0e8cff52bc avformat/whip: add DTLS active role support
Add dtls_active flag to specify the dtls role.

Properly set the send key and recv key depends on DTLS role:

As DTLS server, the recv key is client master key plus salt,
the send key is server master key plus salt.
As DTLS client, the recv key is server master key plus salt,
the send key is client master key plus salt.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-23 10:58:45 +00:00
Jack Lau
b43f8dec18 avformat/whip: add macros to replace magic number
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 14:32:03 +00:00
Jack Lau
bc6164eb6f avformat/whip: remove WHIP_STATE_DTLS_CONNECTING
This value is only useful when dtls handshake is NONBLOCK mode,
dtls handshake just need to call ffurl_handshake once since it
force block mode.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 14:32:03 +00:00
Jack Lau
76b13ca0a6 avformat/whip: check the peer whether is ice lite
See RFC 5245 Section 4.3
If an agent is a lite implementation, it MUST include an "a=ice-lite"
session-level attribute in its SDP.  If an agent is a full
implementation, it MUST NOT include this attribute.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 14:32:03 +00:00
Jack Lau
ec0a04de0d avformat/whip: remind user increase -buffer_size
The udp buffer size might be too small to easily
be full temporarily and return WSAEWOULDBLOCK.
The udp code will handle the windows error code
and convert it to AVERROR(EAGAIN).

This issue just can be reproduced on windows.

If sleep a interval and retry to send pkt when hit
EAGAIN, it will increase latency, and appropriate
interval is hard to define.

So this patch just remind user increase the buffer
size via -buffer_size to avoid this issue.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 09:55:18 +00:00
Jack Lau
b3793d9941 avformat/whip: pass through buffer_size option to udp
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 09:55:18 +00:00
Jack Lau
469aad3897 avformat/whip: add ICE consent freshness support
Refer to RFC 9725 4.2,
"Once a session is set up, consent freshness as per
[RFC7675] SHALL be used to detect non-graceful
disconnection by full ICE implementations and DTLS
teardown for session termination by either side"

Refer to RFC 7675,
send STUN Binding request every 5s,
expire consent after 30s without response,
terminate session when the consent expired.

TODO:
Random consent check interval (4-6s) will be
added later.

Co-authored-by: Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 09:29:25 +00:00
Jack Lau
aba84db96f avformat/whip: add WHIP_US_PER_MS macro to replace 1000
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-10-09 09:29:25 +00:00
Jack Lau
bc561013c9 avformat/whip: add RTX initial support
Refer to RFC 4588.

Add and set the basic param of RTX like
ssrc, payload_type, srtp.

Modify the SDP to add RTX info so that
the peer be able to parse the RTX packet.

There are more pateches to make RTX really
work.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-30 05:42:40 +00:00
Jack Lau
16b8a7805b avformat/whip: enable RTCP NACK for whip
Refer to RFC 4585 4.2,
Add SDP "a=rtcp-fb" so the peer can request rtx
through sending NACKs.

Add basic parsing for NACK packet.
Decrypt into a newly allocated buffer (do not
overwrite whip->buf) so multiple NACKs in a
bundled packet can be parsed in a loop while
keeping whip->buf available for sending rtx

Broken or wrong NACk are logged and skipped.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-15 06:40:11 +00:00
Jack Lau
3c7315c345 avformat/whip: simplify the udp read in whip_write_packet
Reduce nested if statements to make the code structure clearer

Add logic to check if EOF is received from UDP socket.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 04:29:39 +00:00
Jack Lau
785a4e0d6c avformat/whip: fix potential 8bit overflow for profile_idc
The profile contains profile_idc and constraint_set*_flag,
throws away high 8 bit flags and then we get profile_idc.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 04:28:55 +00:00
Jack Lau
6c2768bf8c avformat/whip: fix 8 bits overflow and map constraint_set bits for H264
profile_iop is an 8 bits field. Previous code copied
AVCodecParameters::profile (which can contain bits
beyond 8 bits) into profile_iop, producing overflow
and wrong values.

This patch maps the constrained flags into the proper
profile_iop bits (constraint_set1 / 3)

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 04:28:55 +00:00
Jack Lau
39a63e3d07 avformat/whip: make time measure more precise
Use av_gettime_relative() replace av_gettime() to get
better measure

Changed the display precision from showing only integers
to showing two decimal places (in milliseconds)

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 04:28:24 +00:00
Jack Lau
b41f8207c4 avformat/whip: fix SDP ICE candidates parsing
fix issue #20407

Refer to RFC 5245 15.1,
the foundation may be any string up to 32 chars.
The old code could misread foundations as transport("udp").
This patch fully parse all these attr to avoid parsing error.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 04:27:51 +00:00
Jack Lau
7fd967c2c1 avformat/whip: add PRIORITY and ICE_CONTROLLING for STUN binding request
fix issue #20402

See RFC 8445 7.1,
The PRIORITY and ICE-CONTROLLED or ICE-CONTROLLING must be included
in a Binding request.

FFmpeg is always ICE Controlling role so we use ICE-CONTROLLING attr.

Co-author-by: Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-09-05 01:21:28 +00:00
Jack Lau
4811518143 avformat/whip: set the first sequence number for video and audio
simply whip->audio_first_seq + 1 for video could be
faster than generating a new one.

this patch will make whip be able to get the first
rtp seq easily and compute something sequence-based
(e.g. NACK and RTX handling)

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-08-26 02:36:06 +00:00
Jack Lau
b3ea558492 avformat/whip: fix ssrc might be same
use audio_ssrc + 1 for video_ssrc to avoid ssrc collision

See RFC 3550 Section 8,
The SSRC identifier carried in the RTP header and in various fields
of RTCP packets is a random 32 bit number that is required to be
globally unique within an RTP session.

simply + 1 also can get an unique ssrc, and faster than generating
new unique ssrc.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-08-23 17:20:32 +08:00
Jack Lau
37507c6a78 avformat/whip: remove DTLSState enum
This patch aims to simplify the dtls handshake process
since dtls handshake use force block mode

We can just use the return code instead of DTLSState enum

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-08-16 06:25:04 +00:00
Jack Lau
dc9f676b99 avformat/tls: add new option use_srtp to control whether enable it
The SRTP profile string "SRTP_AES128_CM_SHA1_80" is only used when
use_srtp is enabled. Move its declaration inside the "if (s->use_srtp)"
block to limit scope

Signed-off-by: Jack Lau <jacklau1222@qq.com>
2025-08-09 23:18:57 +02:00
Timo Rothenpieler
262d41c804 all: fix typos found by codespell 2025-08-03 13:48:47 +02:00
Jack Lau
25e710c61e avformat/whip: force NONBLOCK for rtp
We need to ensure rtp sets NONBLOCK since the dtls
handshake has potentially overriden the sockets mode.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2025-07-17 14:33:47 +02:00
Jack Lau
7bd3bdbd62 avformat/whip: free udp socket after dtls free
the SSL_shutdown in tls_close need call the url_bio_bwrite
so we should keep udp still alive

Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2025-07-17 02:35:01 +02:00
Timo Rothenpieler
6af099522e avformat/tls: make passing an external socket universal 2025-07-11 17:49:57 +02:00
Timo Rothenpieler
2c7e117fe0 avformat/tls: move whip specific init out of generic tls code 2025-07-11 17:49:57 +02:00
Timo Rothenpieler
ba984355fe avformat/whip: fix format string for printing size_t 2025-07-03 22:51:15 +02:00
Timo Rothenpieler
3f7e0fddee avformat/tls: clean up new whip options 2025-07-03 22:18:10 +02:00
Timo Rothenpieler
4a4a566d6e avformat/tls: remove unused fingerprint option 2025-07-03 22:18:10 +02:00
Timo Rothenpieler
058b226313 avformat/whip: remove redundant WHIP: prefix from all logging 2025-07-03 22:18:10 +02:00
Timo Rothenpieler
b92130293e avformat/whip: don't leak options dict 2025-07-03 22:18:10 +02:00
Timo Rothenpieler
8f3ecc9db0 avformat/whip: use av_dict_set_int for int 2025-07-03 22:18:10 +02:00
Steven Liu
f8a9d9473b avformat/whip: check the exchange sdp url is start with http
Make sure the WHIP protocol performs the SDP offer/answer
exchange with the WebRTC peer over HTTP.

Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Reviewed-by: Jack Lau <jacklau1222@qq.com>
2025-06-26 22:20:07 +02:00
Steven Liu
87808e38a8 avformat/whip: Remove unnecessary pkt checks
h264_annexb_insert_sps_pps (called after write_packet)
reorganizes PPS, SPS, and IDR packets in H.264 streams.
Since write_packet already validates pkt,
redundant null checks in h264_annexb_insert_sps_pps can be removed.

Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Reviewed-by: Marvin Scholz <epirat07@gmail.com>
2025-06-26 21:46:18 +02:00
Jack Lau
8aa6df663e avformat/whip: replace AV_OPT_FLAG_DECODING_PARAM to ENCODING
Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2025-06-21 12:57:19 +08:00
Jack Lau
214248e2d6 avformat/whip: mark as experimental
This patch doesn't effect WHIP usage via command, as WHIP always
needs to be explicitly specified

Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2025-06-21 12:05:13 +08:00
Zhao Zhili
153cdf3142 avformat/whip: Constify arguments in is_rtp_rtcp/is_rtcp
Fix warning of -Wincompatible-pointer-types-discards-qualifiers.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2025-06-07 15:58:21 +08:00
Jack Lau
167e343bbe avformat/whip: Add WHIP muxer support for subsecond latency streaming
0. WHIP Version 3.
1. The WHIP muxer has been renamed and refined,
    with improved logging context and error messages for SSL, DTLS, and RTC.
2. Magic numbers have been replaced with macros and extracted to functions,
    and log levels have been altered for better clarity.
3. DTLS curve list has been updated,
    and SRTP profile names have been refined for FFmpeg and OpenSSL.
4. ICE STUN magic number has been refined,
    and RTP payload types have been updated based on Chrome's definition.
5. Fixed frame size has been refined to rtc->audio_par->frame_size,
    and h264_mp4toannexb is now used to convert MP4/ISOM to annexb.
6. OPUS timestamp issue has been addressed,
    and marker setting has been corrected after utilizing BSF.
7. DTLS handshake and ICE handling have been optimized for improved performance,
    with a single handshake timeout and server role to prevent ARQ.
8. Consolidated ICE request/response handling and DTLS handshake into a single function,
    and fixed OpenSSL build errors to work with Pion.
9. Merge TLS & DTLS implementation, shared BIO callbacks, read, write,
    print_ssl_error, openssl_init_ca_key_cert,
    init_bio_method function and shared same data structure
10. Modify configure that whip is enabled only dtls is
    enabled(just support openssl for now) to fix build error

Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: yangrtc <yangrtc@aliyun.com>
Co-authored-by: cloudwebrtc <duanweiwei1982@gmail.com>
Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: Steven Liu <lq@chinaffmpeg.org>
Co-authored-by: Jun Zhao <barryjzhao@tencent.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
2025-06-04 11:17:07 +08:00