曾经整理过一个 WebRTC 音频发送和接收处理的关键过程，WebRTC Audio 接收和发送的关键过程 ，不过之前的分析是基于比较老的版本做的。分析所基于的应用程序，依然选择 WebRTC 的示例应用 peerconnection_client。
这里基于 WebRTC 比较新的 M96 版的代码，再来看下音频发送和接收处理过程。
1. 创建 JsepTransportController
#0 webrtc::JsepTransportController::JsepTransportController(rtc::Thread*, cricket::PortAllocator*, webrtc::AsyncDnsResolverFactoryInterface*, webrtc::JsepTransportController::Config) (this=0x0, network_thread=0x0, port_allocator=0x7fffdb3f94f0, async_dns_resolver_factory=0x7fffe4004bc0, config=...)
at ../../pc/jsep_transport_controller.cc:41
#1 0x00005555561bdcc7 in webrtc::PeerConnection::InitializeTransportController_n(webrtc::PeerConnectionInterface::RTCConfiguration const&, webrtc::PeerConnectionDependencies const&) (this=0x7fffe40043c0, configuration=..., dependencies=...) at ../../pc/peer_connection.cc:696
#2 0x00005555561ba922 in webrtc::PeerConnection::<lambda()>::operator()(void) const (__closure=0x7fffeeffc030) at ../../pc/peer_connection.cc:615
#3 0x00005555561e0998 in rtc::FunctionView<void()>::CallVoidPtr<webrtc::PeerConnection::Initialize(const webrtc::PeerConnectionInterface::RTCConfiguration&, webrtc::PeerConnectionDependencies)::<lambda()> >(rtc::FunctionView<void()>::VoidUnion) (vu=...) at ../../api/function_view.h:109
#4 0x000055555605c164 in rtc::FunctionView<void ()>::operator()() const (this=0x7fffeeffbec8) at ../../api/function_view.h:95
2. 创建并注册 transport
#0 webrtc::JsepTransportCollection::RegisterTransport(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<cricket::JsepTransport, std::default_delete<cricket::JsepTransport> >)
(this=0x7fffdb3f8f50, mid="", transport=std::unique_ptr<cricket::JsepTransport> = {...}) at ../../pc/jsep_transport_collection.cc:158
#1 0x0000555556b9e732 in webrtc::JsepTransportController::MaybeCreateJsepTransport(bool, cricket::ContentInfo const&, cricket::SessionDescription const&) (this=0x7fffc0000d10, local=false, content_info=..., description=...) at ../../pc/jsep_transport_controller.cc:1082
#2 0x0000555556b9acae in webrtc::JsepTransportController::ApplyDescription_n(bool, webrtc::SdpType, cricket::SessionDescription const*)
(this=0x7fffc0000d10, local=false, type=webrtc::SdpType::kOffer, description=0x5555586ca200) at ../../pc/jsep_transport_controller.cc:583
#3 0x0000555556b950e7 in webrtc::JsepTransportController::SetRemoteDescription(webrtc::SdpType, cricket::SessionDescription const*)
(this=0x7fffc0000d10, type=webrtc::SdpType::kOffer, description=0x5555586ca200) at ../../pc/jsep_transport_controller.cc:105
#4 0x0000555556b94db6 in webrtc::JsepTransportController::<lambda()>::operator()(void) const (__closure=0x7fffef7fc690)
at ../../pc/jsep_transport_controller.cc:101
#5 0x0000555556ba10cd in rtc::FunctionView<webrtc::RTCError()>::CallVoidPtr<webrtc::JsepTransportController::SetRemoteDescription(webrtc::SdpType, const cricket::SessionDescription*)::<lambda()> >(rtc::FunctionView<webrtc::RTCError()>::VoidUnion) (vu=...) at ../../api/function_view.h:109
3. 获取 RTP transport 并创建 VoiceChannel
在 SdpOfferAnswerHandler::CreateVoiceChannel() 中根据 mid 通过 PeerConnection 获得 RTP transport
cricket::VoiceChannel* SdpOfferAnswerHandler::CreateVoiceChannel(
const std::string& mid) {
TRACE_EVENT0("webrtc", "SdpOfferAnswerHandler::CreateVoiceChannel");
RTC_DCHECK_RUN_ON(signaling_thread());
if (!channel_manager()->media_engine())
return nullptr;
RtpTransportInternal* rtp_transport = pc_->GetRtpTransport(mid);
// TODO(bugs.webrtc.org/11992): CreateVoiceChannel internally switches to the
// worker thread. We shouldn't be using the `call_ptr_` hack here but simply
// be on the worker thread and use `call_` (update upstream code).
return channel_manager()->CreateVoiceChannel(
pc_->call_ptr(), pc_->configuration()->media_config, rtp_transport,
signaling_thread(), mid, pc_->SrtpRequired(), pc_->GetCryptoOptions(),
&ssrc_generator_, audio_options());
}
PeerConnection 将根据 mid 通过 JsepTransportController 获取 RTP transport：
RtpTransportInternal* PeerConnection::GetRtpTransport(const std::string& mid) {
RTC_DCHECK_RUN_ON(signaling_thread());
return network_thread()->Invoke<RtpTransportInternal*>(
RTC_FROM_HERE, [this, &mid] {
auto rtp_transport = transport_controller_->GetRtpTransport(mid);
RTC_DCHECK(rtp_transport);
return rtp_transport;
});
}
然后创建创建 VoiceChannel：
#0 cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)
(this=0x7fffeeffaec0, call=0x5555561f550e <rtc::FunctionView<void ()>::CallVoidPtr<rtc::Thread::Invoke<webrtc::RtpTransportInternal*, void>(rtc::Location const&, rtc::FunctionView<webrtc::RtpTransportInternal* ()>)::{lambda()#1}>(rtc::FunctionView<void ()>::VoidUnion)>, media_config=..., rtp_transport=0x55555825bef0 <vtable for rtc::MessageHandler+16>, signaling_thread=0xd68a928ef375e700, content_name="", srtp_required=255, crypto_options=..., ssrc_generator=0xd68a928ef375e700, options=...)
at ../../pc/channel_manager.cc:150
#1 0x0000555556272a7a in webrtc::SdpOfferAnswerHandler::CreateVoiceChannel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (this=0x7fffe4005350, mid="0") at ../../pc/sdp_offer_answer.cc:4618
#2 0x0000555556263bfb in webrtc::SdpOfferAnswerHandler::UpdateTransceiverChannel(rtc::scoped_refptr<webrtc::RtpTransceiverProxyWithInternal<webrtc::RtpTransceiver> >, cricket::ContentInfo const&, cricket::ContentGroup const*)
(this=0x7fffe4005350, transceiver=..., content=..., bundle_group=0x555558513270) at ../../pc/sdp_offer_answer.cc:3389
#3 0x0000555556260c59 in webrtc::SdpOfferAnswerHandler::UpdateTransceiversAndDataChannels(cricket::ContentSource, webrtc::SessionDescriptionInterface const&, webrtc::SessionDescriptionInterface const*, webrtc::SessionDescriptionInterface const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, cricket::ContentGroup const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, cricket::ContentGroup const*> > > const&)
(this=0x7fffe4005350, source=cricket::CS_REMOTE,
new_session=..., old_local_description=0x0, old_remote_description=0x0, bundle_groups_by_mid=std::map with 2 elements = {...}) at ../../pc/sdp_offer_answer.cc:3209
#4 0x000055555624d8f5 in webrtc::SdpOfferAnswerHandler::ApplyRemoteDescription(std::unique_ptr<webrtc::SessionDescriptionInterface, std::default_delete<webrtc::SessionDescriptionInterface> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, cricket::ContentGroup const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, cricket::ContentGroup const*> > > const&)
(this=0x7fffe4005350, desc=std::unique_ptr<webrtc::SessionDescriptionInterface> = {...}, bundle_groups_by_mid=std::map with 2 elements = {...}) at ../../pc/sdp_offer_answer.cc:1599
#5 0x00005555562546ff in webrtc::SdpOfferAnswerHandler::DoSetRemoteDescription(std::unique_ptr<webrtc::SessionDescriptionInterface, std::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetRemoteDescriptionObserverInterface>) (this=0x7fffe4005350, desc=std::unique_ptr<webrtc::SessionDescriptionInterface> = {...}, observer=...)
at ../../pc/sdp_offer_answer.cc:2196
#6 0x000055555624cbbb in webrtc::SdpOfferAnswerHandler::<lambda(std::function<void()>)>::operator()(std::function<void()>)
(__closure=0x7fffeeffc340, operations_chain_callback=...) at ../../pc/sdp_offer_answer.cc:1510
#7 0x00005555562813b9 in rtc::rtc_operations_chain_internal::OperationWithFunctor<webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)::<lambda(std::function<void()>)> >::Run(void) (this=0x7fffe40185d0) at ../../rtc_base/operations_chain.h:71
#8 0x0000555556278f74 in rtc::OperationsChain::ChainOperation<webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)::<lambda(std::function<void()>)> >(webrtc::SdpOfferAnswerHandler::<lambda(std::function<void()>)> &&) (this=0x7fffe4005670, functor=...) at ../../rtc_base/operations_chain.h:154
#9 0x000055555624ce45 in webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*) (this=0x7fffe4005350, observer=0x55555881d6e0, desc_ptr=0x5555585ca780)
--Type <RET> for more, q to quit, c to continue without paging--
.cc:1494
#10 0x00005555561cc02f in webrtc::PeerConnection::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)
(this=0x7fffe4004580, observer=0x55555881d6e0, desc_ptr=0x5555585ca780) at ../../pc/peer_connection.cc:1362
#11 0x0000555556178768 in webrtc::ReturnType<void>::Invoke<webrtc::PeerConnectionInterface, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>(webrtc::PeerConnectionInterface*, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*&&, webrtc::SessionDescriptionInterface*&&) (this=0x7fffffffc580, c=0x7fffe4004580, m=&virtual table offset 320)
at ../../pc/proxy.h:119
#12 0x000055555617412f in webrtc::MethodCall<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::Invoke<0ul, 1ul>(std::integer_sequence<unsigned long, 0ul, 1ul>) (this=0x7fffffffc560) at ../../pc/proxy.h:153
#13 0x00005555561835b8 in webrtc::MethodCall<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::Run() (this=0x7fffffffc560) at ../../pc/proxy.h:146
#14 0x00005555560575f4 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) (this=0x5555585ace98, msg=0x7fffeeffcab0)
at ../../rtc_base/thread.cc:1042
#15 0x0000555556055398 in rtc::Thread::Dispatch(rtc::Message*) (this=0x5555585acd70, pmsg=0x7fffeeffcab0) at ../../rtc_base/thread.cc:711
ChannelManager::CreateVoiceChannel() 的实现如下：
VoiceChannel* ChannelManager::CreateVoiceChannel(
webrtc::Call* call,
const MediaConfig& media_config,
webrtc::RtpTransportInternal* rtp_transport,
rtc::Thread* signaling_thread,
const std::string& content_name,
bool srtp_required,
const webrtc::CryptoOptions& crypto_options,
rtc::UniqueRandomIdGenerator* ssrc_generator,
const AudioOptions& options) {
RTC_DCHECK(call);
RTC_DCHECK(media_engine_);
// TODO(bugs.webrtc.org/11992): Remove this workaround after updates in
// PeerConnection and add the expectation that we're already on the right
// thread.
if (!worker_thread_->IsCurrent()) {
return worker_thread_->Invoke<VoiceChannel*>(RTC_FROM_HERE, [&] {
return CreateVoiceChannel(call, media_config, rtp_transport,
signaling_thread, content_name, srtp_required,
crypto_options, ssrc_generator, options);
});
}
RTC_DCHECK_RUN_ON(worker_thread_);
VoiceMediaChannel* media_channel = media_engine_->voice().CreateMediaChannel(
call, media_config, options, crypto_options);
if (!media_channel) {
return nullptr;
}
auto voice_channel = std::make_unique<VoiceChannel>(
worker_thread_, network_thread_, signaling_thread,
absl::WrapUnique(media_channel), content_name, srtp_required,
crypto_options, ssrc_generator);
voice_channel->Init_w(rtp_transport);
VoiceChannel* voice_channel_ptr = voice_channel.get();
voice_channels_.push_back(std::move(voice_channel));
return voice_channel_ptr;
}
4. 为 BaseChannel/VoiceChannel 设置 RtpTransport
为 BaseChannel 设置 RtpTransport 的动作在 BaseChannel 中 BaseChannel::Init_w() 函数里的一个 lambda 表达式中完成：
#0 cricket::BaseChannel::SetRtpTransport(webrtc::RtpTransportInternal*)
(this=0x555556ba2534 <cricket::JsepTransport::rtp_transport() const+52>, rtp_transport=0x7fffdb3f9430)
at ../../pc/channel.cc:222
#1 0x0000555556b6f1f6 in cricket::BaseChannel::<lambda()>::operator()(void) const (__closure=0x7fffdabf8300)
at ../../pc/channel.cc:200
#2 0x0000555556b7d400 in rtc::FunctionView<void()>::CallVoidPtr<cricket::BaseChannel::Init_w(webrtc::RtpTransportInternal*)::<lambda()> >(rtc::FunctionView<void()>::VoidUnion) (vu=...) at ../../api/function_view.h:109
BaseChannel::Init_w() 的调用过程如下：
#0 cricket::BaseChannel::Init_w(webrtc::RtpTransportInternal*) (this=0x7fffdabf8328, rtp_transport=0x7fffdabf82c0)
at ../../pc/channel.cc:196
#1 0x0000555556b8dd8a in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)
(this=0x7fffc408c160, call=0x7fffc4090080, media_config=..., rtp_transport=0x7fffc0003e00, signaling_thread=0x5555586bfd90, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe0007710, options=...)
at ../../pc/channel_manager.cc:177
#2 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffef7fbf30)
at ../../pc/channel_manager.cc:158
#3 0x0000555556b8f4dd in rtc::FunctionView<cricket::VoiceChannel*()>::CallVoidPtr<cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, const cricket::MediaConfig&, webrtc::RtpTransportInternal*, rtc::Thread*, const string&, bool, const webrtc::CryptoOptions&, rtc::UniqueRandomIdGenerator*, const cricket::AudioOptions&)::<lambda()> >(rtc::FunctionView<cricket::VoiceChannel*()>::VoidUnion) (vu=...) at ../../api/function_view.h:109
BaseChannel::Init_w() 的代码如下：
void BaseChannel::Init_w(webrtc::RtpTransportInternal* rtp_transport) {
RTC_DCHECK_RUN_ON(worker_thread());
network_thread_->Invoke<void>(RTC_FROM_HERE, [this, rtp_transport] {
SetRtpTransport(rtp_transport);
// Both RTP and RTCP channels should be set, we can call SetInterface on
// the media channel and it can set network options.
media_channel_->SetInterface(this);
});
}
这段代码将数据包发送过程中的多个组件连接起来。MediaChannel::SetInterface(NetworkInterface* iface) 接口需要一个 MediaChannel::NetworkInterface 接口的实现，MediaChannel 可以通过这个接口发送数据包：
class MediaChannel {
public:
class NetworkInterface {
public:
enum SocketType { ST_RTP, ST_RTCP };
virtual bool SendPacket(rtc::CopyOnWriteBuffer* packet,
const rtc::PacketOptions& options) = 0;
virtual bool SendRtcp(rtc::CopyOnWriteBuffer* packet,
const rtc::PacketOptions& options) = 0;
virtual int SetOption(SocketType type,
rtc::Socket::Option opt,
int option) = 0;
virtual ~NetworkInterface() {}
};
BaseChannel 实现 MediaChannel::NetworkInterface 接口。BaseChannel::Init_w() 将 MediaChannel 、BaseChannel 和 RtpTransportInternal 这三个组件连接起来。
5. MediaChannel 是 WebRtcVoiceMediaChannel
BaseChannel 的 MediaChannel 在对象构造的时候传入：
#0 cricket::BaseChannel::BaseChannel(rtc::Thread*, rtc::Thread*, rtc::Thread*, std::unique_ptr<cricket::MediaChannel, std::default_delete<cricket::MediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions, rtc::UniqueRandomIdGenerator*)
(this=0x7fffdabf80e0, worker_thread=0x7fffdabf8100, network_thread=0x7fffdabf80f0, signaling_thread=0x7fffdabf80e0, media_channel=std::unique_ptr<cricket::MediaChannel> = {...}, content_name="", srtp_required=255, crypto_options=..., ssrc_generator=0xc4d93a8ca0272e00) at ../../pc/channel.cc:117
#1 0x0000555556b77d44 in cricket::VoiceChannel::VoiceChannel(rtc::Thread*, rtc::Thread*, rtc::Thread*, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions, rtc::UniqueRandomIdGenerator*)
(this=0x7fffc4092460, worker_thread=0x7fffe4003030, network_thread=0x7fffe4002600, signaling_thread=0x555558805600, media_channel=std::unique_ptr<cricket::VoiceMediaChannel> = {...}, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40077b0) at ../../pc/channel.cc:797
#2 0x0000555556b9001a in std::make_unique<cricket::VoiceChannel, rtc::Thread* const&, rtc::Thread* const&, rtc::Thread*&, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*&>(rtc::Thread* const&, rtc::Thread* const&, rtc::Thread*&, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*&) () at /usr/include/c++/9/bits/unique_ptr.h:857
#3 0x0000555556b8dd4f in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)
(this=0x7fffc408c160, call=0x7fffc4090090, media_config=..., rtp_transport=0x7fffc0003f10, signaling_thread=0x555558805600, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40077b0, options=...) at ../../pc/channel_manager.cc:175
#4 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffeeffaf30) at ../../pc/channel_manager.cc:158
ChannelManager 中通过 WebRtcVoiceEngine/VoiceEngine 创建 VoiceChannel/BaseChannel 的过程如下：
VoiceChannel* ChannelManager::CreateVoiceChannel(
webrtc::Call* call,
const MediaConfig& media_config,
webrtc::RtpTransportInternal* rtp_transport,
rtc::Thread* signaling_thread,
const std::string& content_name,
bool srtp_required,
const webrtc::CryptoOptions& crypto_options,
rtc::UniqueRandomIdGenerator* ssrc_generator,
const AudioOptions& options) {
RTC_DCHECK(call);
RTC_DCHECK(media_engine_);
// TODO(bugs.webrtc.org/11992): Remove this workaround after updates in
// PeerConnection and add the expectation that we're already on the right
// thread.
if (!worker_thread_->IsCurrent()) {
return worker_thread_->Invoke<VoiceChannel*>(RTC_FROM_HERE, [&] {
return CreateVoiceChannel(call, media_config, rtp_transport,
signaling_thread, content_name, srtp_required,
crypto_options, ssrc_generator, options);
});
}
RTC_DCHECK_RUN_ON(worker_thread_);
VoiceMediaChannel* media_channel = media_engine_->voice().CreateMediaChannel(
call, media_config, options, crypto_options);
if (!media_channel) {
return nullptr;
}
auto voice_channel = std::make_unique<VoiceChannel>(
worker_thread_, network_thread_, signaling_thread,
absl::WrapUnique(media_channel), content_name, srtp_required,
crypto_options, ssrc_generator);
voice_channel->Init_w(rtp_transport);
VoiceChannel* voice_channel_ptr = voice_channel.get();
voice_channels_.push_back(std::move(voice_channel));
return voice_channel_ptr;
}
MediaChannel/VoiceMediaChannel 是 WebRtcVoiceMediaChannel:
#0 webrtc::internal::Call::worker_thread() const (this=0x5555557d146f <webrtc::MutexLock::~MutexLock()+31>) at ../../call/call.cc:1285
#1 0x00005555560d9e8f in cricket::WebRtcVoiceEngine::CreateMediaChannel(webrtc::Call*, cricket::MediaConfig const&, cricket::AudioOptions const&, webrtc::CryptoOptions const&) (this=0x55555844a800, call=0x7fffc4090090, config=..., options=..., crypto_options=...)
at ../../media/engine/webrtc_voice_engine.cc:432
#2 0x0000555556b8dcc1 in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)
(this=0x7fffc408c160, call=0x7fffc4090090, media_config=..., rtp_transport=0x7fffc0003f20, signaling_thread=0x55555861bdd0, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40055b0, options=...) at ../../pc/channel_manager.cc:166
#3 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffeeffaf50) at ../../pc/channel_manager.cc:158
#4 0x0000555556b8f4dd in rtc::FunctionView<cricket::VoiceChannel*()>::CallVoidPtr<cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, const cricket::MediaConfig&, webrtc::RtpTransportInternal*, rtc::Thread*, const string&, bool, const webrtc::CryptoOptions&, rtc::UniqueRandomIdGenerator*, const cricket::AudioOptions&)::<lambda()> >(rtc::FunctionView<cricket::VoiceChannel*()>::VoidUnion) (vu=...) at ../../api/function_view.h:109
6. 音频数据包的发送处理
音频数据包的编码发送，分为几个阶段。
第一阶段，录制线程将录制的麦克风音频数据抛上来，送进编码线程。
#0 webrtc::voe::(anonymous namespace)::ChannelSend::ProcessAndEncodeAudio(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >) (this=0x5555561264fa <webrtc::AudioProcessingImpl::ApmStatsReporter::GetStatistics()+106>, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...})
at ../../audio/channel_send.cc:815
#1 0x000055555698a90e in webrtc::internal::AudioSendStream::SendAudioData(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >) (this=0x7fffc40dab90, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...}) at ../../audio/audio_send_stream.cc:422
#2 0x000055555699e2e4 in webrtc::AudioTransportImpl::SendProcessedData(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >)
(this=0x7fffc4024f60, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...}) at ../../audio/audio_transport_impl.cc:190
#3 0x000055555699df46 in webrtc::AudioTransportImpl::RecordedDataIsAvailable(void const*, unsigned long, unsigned long, unsigned long, unsigned int, unsigned int, int, unsigned int, bool, unsigned int&)
(this=0x7fffc4024f60, audio_data=0x7fff84001030, number_of_frames=441, bytes_per_sample=4, number_of_channels=2, sample_rate=44100, audio_delay_milliseconds=207, key_pressed=false) at ../../audio/audio_transport_impl.cc:171
#4 0x00005555565b7862 in webrtc::AudioDeviceBuffer::DeliverRecordedData() (this=0x7fffc4002c28)
at ../../modules/audio_device/audio_device_buffer.cc:270
#5 0x00005555565a90c0 in webrtc::AudioDeviceLinuxPulse::ProcessRecordedData(signed char*, unsigned int, unsigned int)
(this=0x7fffc4004de0, bufferData=0x7fffb0910374 "\r", bufferSizeInSamples=441, recDelay=10)
at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:1968
#6 0x00005555565a8f80 in webrtc::AudioDeviceLinuxPulse::ReadRecordedData(void const*, unsigned long)
(this=0x7fffc4004de0, bufferData=0x7fffb0910374, bufferSize=4408) at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:1926
#7 0x00005555565abcc1 in webrtc::AudioDeviceLinuxPulse::RecThreadProcess() (this=0x7fffc4004de0)
at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:2247
#8 0x000055555659e509 in webrtc::AudioDeviceLinuxPulse::<lambda()>::operator()(void) const (__closure=0x7fffc4003740)
at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:166
#9 0x00005555565abf53 in std::_Function_handler<void(), webrtc::AudioDeviceLinuxPulse::Init()::<lambda()> >::_M_invoke(const std::_Any_data &)
(__functor=...) at /usr/include/c++/9/bits/std_function.h:300
第二阶段，编码线程将编码的数据送进 PacedSender 的队列里。
#0 webrtc::PacingController::EnqueuePacket(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >)
(this=0x55555692fb9a <std::tuple<webrtc::RtpPacketToSend*, std::default_delete<webrtc::RtpPacketToSend> >::tuple<webrtc::RtpPacketToSend*&, std::default_delete<webrtc::RtpPacketToSend>, true>(webrtc::RtpPacketToSend*&, std::default_delete<webrtc::RtpPacketToSend>&&)+72>, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}) at ../../modules/pacing/pacing_controller.cc:235
#1 0x000055555692edbc in webrtc::PacedSender::EnqueuePackets(std::vector<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, std::allocator<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> > > >)
(this=0x7fffc408e530, packets=std::vector of length 1, capacity 1 = {...}) at ../../modules/pacing/paced_sender.cc:128
#2 0x00005555569b1a6b in webrtc::voe::(anonymous namespace)::RtpPacketSenderProxy::EnqueuePackets(std::vector<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, std::allocator<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> > > >) (this=0x7fffc40db280, packets=std::vector of length 0, capacity 0) at ../../audio/channel_send.cc:270
#3 0x0000555557425210 in webrtc::RTPSender::SendToNetwork(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >)
(this=0x7fffc40f35d0, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}) at ../../modules/rtp_rtcp/source/rtp_sender.cc:499
#4 0x000055555742b1bb in webrtc::RTPSenderAudio::SendAudio(webrtc::AudioFrameType, signed char, unsigned int, unsigned char const*, unsigned long, long)
(this=0x7fffc40f4870, frame_type=webrtc::AudioFrameType::kAudioFrameSpeech, payload_type=111 'o', rtp_timestamp=1692995353, payload_data=0x7fff9c000b90 "x\v\350A1[\355\301eם\257~-rs\252\261\206\314\333X\315\"N}\022\251y\235\352X\234\021x\223\260\206\261\255\243wR7T\317\353\177\035\366\211", <incomplete sequence \362\274>, payload_size=56, absolute_capture_timestamp_ms=-1) at ../../modules/rtp_rtcp/source/rtp_sender_audio.cc:316
#5 0x00005555569b27a4 in webrtc::voe::(anonymous namespace)::ChannelSend::SendRtpAudio(webrtc::AudioFrameType, uint8_t, uint32_t, rtc::ArrayView<unsigned char const, -4711>, int64_t)
(this=0x7fffc40db010, frameType=webrtc::AudioFrameType::kAudioFrameSpeech, payloadType=111 'o', rtp_timestamp=0, payload=..., absolute_capture_timestamp_ms=-1) at ../../audio/channel_send.cc:442
#6 0x00005555569b20f7 in webrtc::voe::(anonymous namespace)::ChannelSend::SendData(webrtc::AudioFrameType, uint8_t, uint32_t, uint8_t const*, size_t, int64_t)
(this=0x7fffc40db010, frameType=webrtc::AudioFrameType::kAudioFrameSpeech, payloadType=111 'o', rtp_timestamp=0, payloadData=0x7fff9c000b90 "x\v\350A1[\355\301eם\257~-rs\252\261\206\314\333X\315\"N}\022\251y\235\352X\234\021x\223\260\206\261\255\243wR7T\317\353\177\035\366\211", <incomplete sequence \362\274>, payloadSize=56, absolute_capture_timestamp_ms=-1) at ../../audio/channel_send.cc:370
#7 0x00005555569c9b0c in webrtc::(anonymous namespace)::AudioCodingModuleImpl::Encode(webrtc::(anonymous namespace)::AudioCodingModuleImpl::InputData const&, absl::optional<long>) (this=0x7fffc40dca00, input_data=..., absolute_capture_timestamp_ms=...)
at ../../modules/audio_coding/acm2/audio_coding_module.cc:302
#8 0x00005555569c9c9d in webrtc::(anonymous namespace)::AudioCodingModuleImpl::Add10MsData(webrtc::AudioFrame const&)
(this=0x7fffc40dca00, audio_frame=...) at ../../modules/audio_coding/acm2/audio_coding_module.cc:339
#9 0x00005555569b8269 in webrtc::voe::(anonymous namespace)::ChannelSend::<lambda()>::operator()(void) (__closure=0x7fff880e1d98)
at ../../audio/channel_send.cc:864
#10 0x00005555569bd044 in webrtc::webrtc_new_closure_impl::ClosureTask<webrtc::voe::(anonymous namespace)::ChannelSend::ProcessAndEncodeAudio(std::unique_ptr<webrtc::AudioFrame>)::<lambda()> >::Run(void) (this=0x7fff880e1d90) at ../../rtc_base/task_utils/to_queued_task.h:32
第三阶段，PacedSender 将 RTP 包通过 MediaChannel 发送出去。
#0 cricket::MediaChannel::SendRtp(unsigned char const*, unsigned long, webrtc::PacketOptions const&)
(this=0x7fff9c001648, data=0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>, len=93825004092765, options=...)
at ../../media/base/media_channel.cc:169
#1 0x00005555560f11dc in cricket::WebRtcVoiceMediaChannel::SendRtp(unsigned char const*, unsigned long, webrtc::PacketOptions const&)
(this=0x7fffc4091a90, data=0x7fff9c0016d0 "\220\357\061&d\351\017\031\342f\335\\\276", <incomplete sequence \336>, len=84, options=...)
at ../../media/engine/webrtc_voice_engine.cc:2572
#2 0x0000555557430fd4 in webrtc::RtpSenderEgress::SendPacketToNetwork(webrtc::RtpPacketToSend const&, webrtc::PacketOptions const&, webrtc::PacedPacketInfo const&) (this=0x7fffc40f3310, packet=..., options=..., pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_sender_egress.cc:555
#3 0x000055555742e285 in webrtc::RtpSenderEgress::SendPacket(webrtc::RtpPacketToSend*, webrtc::PacedPacketInfo const&)
(this=0x7fffc40f3310, packet=0x7fff9c0015f0, pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_sender_egress.cc:273
#4 0x000055555741cc89 in webrtc::ModuleRtpRtcpImpl2::TrySendPacket(webrtc::RtpPacketToSend*, webrtc::PacedPacketInfo const&)
(this=0x7fffc40f2800, packet=0x7fff9c0015f0, pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_rtcp_impl2.cc:376
#5 0x0000555556936fd5 in webrtc::PacketRouter::SendPacket(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, webrtc::PacedPacketInfo const&) (this=0x7fffc408df18, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}, cluster_info=...)
at ../../modules/pacing/packet_router.cc:160
#6 0x00005555569347ca in webrtc::PacingController::ProcessPackets() (this=0x7fffc408e598) at ../../modules/pacing/pacing_controller.cc:590
#7 0x000055555692f1e3 in webrtc::PacedSender::Process() (this=0x7fffc408e530) at ../../modules/pacing/paced_sender.cc:183
#8 0x000055555692f6cb in webrtc::PacedSender::ModuleProxy::Process() (this=0x7fffc408e548) at ../../modules/pacing/paced_sender.h:152
#9 0x00005555573aaafe in webrtc::ProcessThreadImpl::Process() (this=0x7fffc408daa0) at ../../modules/utility/source/process_thread_impl.cc:257
#10 0x00005555573a8e9b in webrtc::ProcessThreadImpl::<lambda()>::operator()(void) const (__closure=0x7fffc4095e50)
at ../../modules/utility/source/process_thread_impl.cc:86
MediaChannel::SendRtp() 会将发送转到网络发送线程中去。
void MediaChannel::SendRtp(const uint8_t* data,
size_t len,
const webrtc::PacketOptions& options) {
auto send =
[this, packet_id = options.packet_id,
included_in_feedback = options.included_in_feedback,
included_in_allocation = options.included_in_allocation,
packet = rtc::CopyOnWriteBuffer(data, len, kMaxRtpPacketLen)]() mutable {
rtc::PacketOptions rtc_options;
rtc_options.packet_id = packet_id;
if (DscpEnabled()) {
rtc_options.dscp = PreferredDscp();
}
rtc_options.info_signaled_after_sent.included_in_feedback =
included_in_feedback;
rtc_options.info_signaled_after_sent.included_in_allocation =
included_in_allocation;
SendPacket(&packet, rtc_options);
};
// TODO(bugs.webrtc.org/11993): ModuleRtpRtcpImpl2 and related classes (e.g.
// RTCPSender) aren't aware of the network thread and may trigger calls to
// this function from different threads. Update those classes to keep
// network traffic on the network thread.
if (network_thread_->IsCurrent()) {
send();
} else {
network_thread_->PostTask(ToQueuedTask(network_safety_, std::move(send)));
}
}
第四阶段，通过 socket 接口将数据包发送到网络。
#0 rtc::PhysicalSocket::DoSendTo(int, char const*, int, int, sockaddr const*, unsigned int)
(this=0x5555563da778 <rtc::SocketAddress::ToSockAddrStorage(sockaddr_storage*) const+58>, socket=32767, buf=0x7fffdb3f8770 "\001\001", len=-616594960, flags=0, dest_addr=0x0, addrlen=16) at ../../rtc_base/physical_socket_server.cc:509
#1 0x00005555560463f3 in rtc::PhysicalSocket::SendTo(void const*, unsigned long, rtc::SocketAddress const&)
(this=0x7fffc00069f8, buffer=0x7fffe0003560, length=100, addr=...) at ../../rtc_base/physical_socket_server.cc:375
#2 0x0000555557346e8e in rtc::AsyncUDPSocket::SendTo(void const*, unsigned long, rtc::SocketAddress const&, rtc::PacketOptions const&)
(this=0x7fffc0005210, pv=0x7fffe0003560, cb=100, addr=..., options=...) at ../../rtc_base/async_udp_socket.cc:84
#3 0x000055555731241a in cricket::UDPPort::SendTo(void const*, unsigned long, rtc::SocketAddress const&, rtc::PacketOptions const&, bool)
(this=0x7fffc00196a0, data=0x7fffe0003560, size=100, addr=..., options=..., payload=true) at ../../p2p/base/stun_port.cc:286
#4 0x000055555730d814 in cricket::ProxyConnection::Send(void const*, unsigned long, rtc::PacketOptions const&)
(this=0x7fffc001a7f0, data=0x7fffe0003560, size=100, options=...) at ../../p2p/base/connection.cc:1371
#5 0x000055555728dfac in cricket::P2PTransportChannel::SendPacket(char const*, unsigned long, rtc::PacketOptions const&, int)
(this=0x7fffc0002b20, data=0x7fffe0003560 "\220o1Id\351\222Y\342f\335\\\276", <incomplete sequence \336>, len=100, options=..., flags=0)
at ../../p2p/base/p2p_transport_channel.cc:1616
#6 0x000055555726e7de in cricket::DtlsTransport::SendPacket(char const*, unsigned long, rtc::PacketOptions const&, int)
(this=0x7fffc00033c0, data=0x7fffe0003560 "\220o1Id\351\222Y\342f\335\\\276", <incomplete sequence \336>, size=100, options=..., flags=1)
at ../../p2p/base/dtls_transport.cc:417
#7 0x0000555556bd5522 in webrtc::RtpTransport::SendPacket(bool, rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&, int)
(this=0x7fffc0003e00, rtcp=false, packet=0x7fffe00154e8, options=..., flags=1) at ../../pc/rtp_transport.cc:147
#8 0x0000555556bdf39c in webrtc::SrtpTransport::SendRtpPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&, int)
(this=0x7fffc0003e00, packet=0x7fffe00154e8, options=..., flags=1) at ../../pc/srtp_transport.cc:173
#9 0x0000555556b731d8 in cricket::BaseChannel::SendPacket(bool, rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)
(this=0x7fffc4092460, rtcp=false, packet=0x7fffe00154e8, options=...) at ../../pc/channel.cc:437
#10 0x0000555556b710c9 in cricket::BaseChannel::SendPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)
(this=0x7fffc4092460, packet=0x7fffe00154e8, options=...) at ../../pc/channel.cc:318
#11 0x0000555557257ea2 in cricket::MediaChannel::DoSendPacket(rtc::CopyOnWriteBuffer*, bool, rtc::PacketOptions const&)
(this=0x7fffc4091a90, packet=0x7fffe00154e8, rtcp=false, options=...) at ../../media/base/media_channel.cc:163
#12 0x0000555557257496 in cricket::MediaChannel::SendPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)
(this=0x7fffc4091a90, packet=0x7fffe00154e8, options=...) at ../../media/base/media_channel.cc:71
#13 0x0000555557257f9a in cricket::MediaChannel::<lambda()>::operator()(void) (__closure=0x7fffe00154d8) at ../../media/base/media_channel.cc:184
#14 0x000055555725aeb2 in webrtc::webrtc_new_closure_impl::SafetyClosureTask<cricket::MediaChannel::SendRtp(const uint8_t*, size_t, const webrtc::PacketOptions&)::<lambda()> >::Run(void) (this=0x7fffe00154d0) at ../../rtc_base/task_utils/to_queued_task.h:50
PhysicalSocket::DoSendTo() 实现如下：
int PhysicalSocket::DoSendTo(SOCKET socket,
const char* buf,
int len,
int flags,
const struct sockaddr* dest_addr,
socklen_t addrlen) {
return ::sendto(socket, buf, len, flags, dest_addr, addrlen);
}
在 AudioTransportImpl::RecordedDataIsAvailable() 中有如下这段代码：
RTC_DCHECK_GT(audio_frame->samples_per_channel_, 0);
if (async_audio_processing_)
async_audio_processing_->Process(std::move(audio_frame));
else
SendProcessedData(std::move(audio_frame));
return 0;
}
这也就意味着，在一些情况下，音频发送过程会多转一次线程。
7. 音频数据包的接收处理
音频数据包的接收处理分为几个阶段。
第一阶段，从网络接收音频 RTP 包。
#0 cricket::WebRtcVoiceMediaChannel::OnPacketReceived(rtc::CopyOnWriteBuffer, long)
(this=0x5555557d7c73 <rtc::dchecked_cast<long, long>(long)+28>, packet=..., packet_time_us=1640606419828691)
at ../../media/engine/webrtc_voice_engine.cc:2217
#1 0x0000555556b736fe in cricket::BaseChannel::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc4092460, parsed_packet=...)
at ../../pc/channel.cc:467
#2 0x00005555568bec90 in webrtc::RtpDemuxer::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc00040a8, packet=...)
at ../../call/rtp_demuxer.cc:249
#3 0x0000555556bd5d12 in webrtc::RtpTransport::DemuxPacket(rtc::CopyOnWriteBuffer, long)
(this=0x7fffc0003f20, packet=..., packet_time_us=1640606419828691) at ../../pc/rtp_transport.cc:194
#4 0x0000555556be0551 in webrtc::SrtpTransport::OnRtpPacketReceived(rtc::CopyOnWriteBuffer, long)
(this=0x7fffc0003f20, packet=..., packet_time_us=1640606419828691) at ../../pc/srtp_transport.cc:226
#5 0x0000555556bd6898 in webrtc::RtpTransport::OnReadPacket(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc0003f20, transport=0x7fffc00034e0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, len=106, packet_time_us=@0x7fffdb3f8bc8: 1640606419828691, flags=1) at ../../pc/rtp_transport.cc:268
#6 0x0000555556bd7d41 in sigslot::_opaque_connection::emitter<webrtc::RtpTransport, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(sigslot::_opaque_connection const*, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) (self=0x7fffc0004420)
at ../../rtc_base/third_party/sigslot/sigslot.h:342
#7 0x0000555557276ec7 in sigslot::_opaque_connection::emit<rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) const (this=0x7fffc0004420) at ../../rtc_base/third_party/sigslot/sigslot.h:331
#8 0x000055555727656d in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::emit(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc00035d8, args#0=0x7fffc00034e0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=1) at ../../rtc_base/third_party/sigslot/sigslot.h:566
#9 0x0000555557275b90 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::operator()(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc00035d8, args#0=0x7fffc00034e0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=1) at ../../rtc_base/third_party/sigslot/sigslot.h:570
#10 0x000055555727172e in cricket::DtlsTransport::OnReadPacket(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc00034e0, transport=0x7fffc0002c40, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, size=106, packet_time_us=@0x7fffdb3f8bc8: 1640606419828691, flags=0) at ../../p2p/base/dtls_transport.cc:627
#11 0x0000555557276d64 in sigslot::_opaque_connection::emitter<cricket::DtlsTransport, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(sigslot::_opaque_connection const*, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) (self=0x7fffc0002340)
at ../../rtc_base/third_party/sigslot/sigslot.h:342
#12 0x0000555557276ec7 in sigslot::_opaque_connection::emit<rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) const (this=0x7fffc0002340) at ../../rtc_base/third_party/sigslot/sigslot.h:331
#13 0x000055555727656d in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::emit(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc0002d38, args#0=0x7fffc0002c40, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=0) at ../../rtc_base/third_party/sigslot/sigslot.h:566
#14 0x0000555557275b90 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::operator()(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)
(this=0x7fffc0002d38, args#0=0x7fffc0002c40, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, ar--Type <RET> for more, q to quit, c to continue without paging--
gs#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=0) at ../../rtc_base/third_party/sigslot/sigslot.h:570
#15 0x0000555557295a6d in cricket::P2PTransportChannel::OnReadPacket(cricket::Connection*, char const*, unsigned long, long)
(this=0x7fffc0002c40, connection=0x7fffc0031650, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, len=106, packet_time_us=1640606419828691) at ../../p2p/base/p2p_transport_channel.cc:2228
#16 0x00005555572a4e14 in sigslot::_opaque_connection::emitter<cricket::P2PTransportChannel, cricket::Connection*, char const*, unsigned long, long>(sigslot::_opaque_connection const*, cricket::Connection*, char const*, unsigned long, long) (self=0x7fffc40ccb20)
at ../../rtc_base/third_party/sigslot/sigslot.h:342
#17 0x000055555730fcb0 in sigslot::_opaque_connection::emit<cricket::Connection*, char const*, unsigned long, long>(cricket::Connection*, char const*, unsigned long, long) const (this=0x7fffc40ccb20) at ../../rtc_base/third_party/sigslot/sigslot.h:331
#18 0x000055555730f612 in sigslot::signal_with_thread_policy<sigslot::single_threaded, cricket::Connection*, char const*, unsigned long, long>::emit(cricket::Connection*, char const*, unsigned long, long)
(this=0x7fffc0031720, args#0=0x7fffc0031650, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:566
#19 0x000055555730ed67 in sigslot::signal_with_thread_policy<sigslot::single_threaded, cricket::Connection*, char const*, unsigned long, long>::operator()(cricket::Connection*, char const*, unsigned long, long)
(this=0x7fffc0031720, args#0=0x7fffc0031650, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:570
#20 0x00005555573004c7 in cricket::Connection::OnReadPacket(char const*, unsigned long, long)
(this=0x7fffc0031650, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, size=106, packet_time_us=1640606419828691) at ../../p2p/base/connection.cc:465
#21 0x00005555573134b8 in cricket::UDPPort::OnReadPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)
(this=
0x7fffc002aa60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=@0x7fffdb3f9260: 1640606419828691) at ../../p2p/base/stun_port.cc:394
#22 0x0000555557312fa4 in cricket::UDPPort::HandleIncomingPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long)
(this=0x7fffc002aa60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=1640606419828691) at ../../p2p/base/stun_port.cc:335
#23 0x00005555572d854f in cricket::AllocationSequence::OnReadPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)
(this=0x7fffc0009e60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=@0x7fffdb3f9600: 1640606419828691) at ../../p2p/client/basic_port_allocator.cc:1641
#24 0x00005555572e857e in sigslot::_opaque_connection::emitter<cricket::AllocationSequence, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>(sigslot::_opaque_connection const*, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&) (self=0x7fffc001a1c0) at ../../rtc_base/third_party/sigslot/sigslot.h:342
#25 0x00005555572eff15 in sigslot::_opaque_connection::emit<rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&) const (this=0x7fffc001a1c0)
at ../../rtc_base/third_party/sigslot/sigslot.h:331
#26 0x00005555572efda3 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>::emit(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)
(this=0x7fffc0009ff0, args#0=0x7fffc0009fa0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=..., args#4=@0x7fffdb3f9600: 1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:566
#27 0x00005555572efbe0 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketA--Type <RET> for more, q to quit, c to continue without paging--
ddress const&, long const&>::operator()(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)
(this=0x7fffc0009ff0, args#0=0x7fffc0009fa0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~ߔ\276", <incomplete sequence \336>, args#2=106, args#3=..., args#4=@0x7fffdb3f9600: 1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:570
#28 0x00005555573475ff in rtc::AsyncUDPSocket::OnReadEvent(rtc::Socket*) (this=0x7fffc0009fa0, socket=0x7fffc00096e8)
at ../../rtc_base/async_udp_socket.cc:132
#29 0x00005555573477e9 in sigslot::_opaque_connection::emitter<rtc::AsyncUDPSocket, rtc::Socket*>(sigslot::_opaque_connection const*, rtc::Socket*)
(self=0x7fffc0009b70) at ../../rtc_base/third_party/sigslot/sigslot.h:342
#30 0x000055555604eb9d in sigslot::_opaque_connection::emit<rtc::Socket*>(rtc::Socket*) const (this=0x7fffc0009b70)
at ../../rtc_base/third_party/sigslot/sigslot.h:331
#31 0x000055555604d8e6 in sigslot::signal_with_thread_policy<sigslot::multi_threaded_local, rtc::Socket*>::emit(rtc::Socket*)
(this=0x7fffc00096f0, args#0=0x7fffc00096e8) at ../../rtc_base/third_party/sigslot/sigslot.h:566
#32 0x000055555604ccb5 in sigslot::signal_with_thread_policy<sigslot::multi_threaded_local, rtc::Socket*>::operator()(rtc::Socket*)
(this=0x7fffc00096f0, args#0=0x7fffc00096e8) at ../../rtc_base/third_party/sigslot/sigslot.h:570
#33 0x0000555556047f11 in rtc::SocketDispatcher::OnEvent(unsigned int, int) (this=0x7fffc00096e0, ff=1, err=0)
at ../../rtc_base/physical_socket_server.cc:831
#34 0x00005555560492d4 in rtc::ProcessEvents(rtc::Dispatcher*, bool, bool, bool)
(dispatcher=0x7fffc00096e0, readable=true, writable=false, check_error=false) at ../../rtc_base/physical_socket_server.cc:1222
#35 0x000055555604b232 in rtc::PhysicalSocketServer::WaitEpoll(int) (this=0x7fffe4002cd0, cmsWait=42) at ../../rtc_base/physical_socket_server.cc:1454
#36 0x0000555556049193 in rtc::PhysicalSocketServer::Wait(int, bool) (this=0x7fffe4002cd0, cmsWait=42, process_io=true)
at ../../rtc_base/physical_socket_server.cc:1169
#37 0x00005555560546a0 in rtc::Thread::Get(rtc::Message*, int, bool) (this=0x7fffe4001820, pmsg=0x7fffdb3f9ab0, cmsWait=-1, process_io=true)
at ../../rtc_base/thread.cc:547
第二阶段，将接收到的音频 RTP 包异步插入 NetEQ 的包缓冲区里。
#0 webrtc::NetEqImpl::InsertPacketInternal(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>)
(this=0x7fffc0094a80, rtp_header=..., payload=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:563
#1 0x00005555569d7cbb in webrtc::NetEqImpl::InsertPacket(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>) (this=0x7fffc0094a80, rtp_header=..., payload=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:192
#2 0x00005555569c5425 in webrtc::acm2::AcmReceiver::InsertPacket(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>) (this=0x7fffc009a958, rtp_header=..., incoming_payload=...) at ../../modules/audio_coding/acm2/acm_receiver.cc:136
#3 0x00005555569a0e4a in webrtc::voe::(anonymous namespace)::ChannelReceive::OnReceivedPayloadData(rtc::ArrayView<unsigned char const, -4711>, webrtc::RTPHeader const&) (this=0x7fffc009a790, payload=..., rtpHeader=...)
at ../../audio/channel_receive.cc:340
#4 0x00005555569a48a3 in webrtc::voe::(anonymous namespace)::ChannelReceive::ReceivePacket(uint8_t const*, size_t, webrtc::RTPHeader const&)
(this=0x7fffc009a790, packet=0x7fffe0069600 "\220o\003\227\372l\023\347֥A(\276", <incomplete sequence \336>, packet_length=80, header=...) at ../../audio/channel_receive.cc:719
#5 0x00005555569a41bc in webrtc::voe::(anonymous namespace)::ChannelReceive::OnRtpPacket(webrtc::RtpPacketReceived const&)
(this=0x7fffc009a790, packet=...) at ../../audio/channel_receive.cc:669
#6 0x00005555568bec90 in webrtc::RtpDemuxer::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc0090340, packet=...)
at ../../call/rtp_demuxer.cc:249
#7 0x00005555568bae55 in webrtc::RtpStreamReceiverController::OnRtpPacket(webrtc::RtpPacketReceived const&)
(this=0x7fffc00902e0, packet=...) at ../../call/rtp_stream_receiver_controller.cc:52
#8 0x0000555556879062 in webrtc::internal::Call::DeliverRtp(webrtc::MediaType, rtc::CopyOnWriteBuffer, long)
(this=0x7fffc0090090, media_type=webrtc::MediaType::AUDIO, packet=..., packet_time_us=1640601570497083)
at ../../call/call.cc:1587
#9 0x0000555556879641 in webrtc::internal::Call::DeliverPacket(webrtc::MediaType, rtc::CopyOnWriteBuffer, long)
(this=0x7fffc0090090, media_type=webrtc::MediaType::AUDIO, packet=..., packet_time_us=1640601570497083)
at ../../call/call.cc:1618
#10 0x00005555560ebede in cricket::WebRtcVoiceMediaChannel::<lambda()>::operator()(void) const (__closure=0x7fffe005a768)
at ../../media/engine/webrtc_voice_engine.cc:2229
#11 0x00005555560f2b74 in webrtc::webrtc_new_closure_impl::SafetyClosureTask<cricket::WebRtcVoiceMediaChannel::OnPacketReceived(rtc::CopyOnWriteBuffer, int64_t)::<lambda()> >::Run(void) (this=0x7fffe005a760)
at ../../rtc_base/task_utils/to_queued_task.h:50
#12 0x00005555560575f4 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) (this=0x7fffe4002728, msg=0x7fffdb3f9ab0)
at ../../rtc_base/thread.cc:1042
#13 0x0000555556055398 in rtc::Thread::Dispatch(rtc::Message*) (this=0x7fffe4002600, pmsg=0x7fffdb3f9ab0)
at ../../rtc_base/thread.cc:711
#14 0x000055555605842a in rtc::Thread::ProcessMessages(int) (this=0x7fffe4002600, cmsLoop=-1) at ../../rtc_base/thread.cc:1149
#15 0x000055555605694f in rtc::Thread::Run() (this=0x7fffe4002600) at ../../rtc_base/thread.cc:901
#16 0x0000555556056914 in rtc::Thread::PreRun(void*) (pv=0x7fffe4002600) at ../../rtc_base/thread.cc:890
#17 0x00007ffff7e55609 in start_thread (arg=<optimized out>) at pthread_create.c:477
第三阶段，播放线程从 NetEQ 里获取音频包，解码并播放出来。
#0 webrtc::NetEqImpl::GetAudioInternal(webrtc::AudioFrame*, bool*, absl::optional<webrtc::NetEq::Operation>)
(this=0x0, audio_frame=0x0, muted=0x0, action_override=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:836
#1 0x00005555569d804f in webrtc::NetEqImpl::GetAudio(webrtc::AudioFrame*, bool*, int*, absl::optional<webrtc::NetEq::Operation>)
(this=0x7fffc4094ab0, audio_frame=0x7fffc4100b60, muted=0x7fffcd9f3aa7, current_sample_rate_hz=0x7fffcd9efc40, action_override=...)
at ../../modules/audio_coding/neteq/neteq_impl.cc:261
#2 0x00005555569c5896 in webrtc::acm2::AcmReceiver::GetAudio(int, webrtc::AudioFrame*, bool*)
(this=0x7fffc409a958, desired_freq_hz=16000, audio_frame=0x7fffc4100b60, muted=0x7fffcd9f3aa7)
at ../../modules/audio_coding/acm2/acm_receiver.cc:151
#3 0x00005555569a1d8b in webrtc::voe::(anonymous namespace)::ChannelReceive::GetAudioFrameWithInfo(int, webrtc::AudioFrame*)
(this=0x7fffc409a790, sample_rate_hz=16000, audio_frame=0x7fffc4100b60) at ../../audio/channel_receive.cc:388
#4 0x0000555556984783 in webrtc::internal::AudioReceiveStream::GetAudioFrameWithInfo(int, webrtc::AudioFrame*)
(this=0x7fffc409a530, sample_rate_hz=16000, audio_frame=0x7fffc4100b60) at ../../audio/audio_receive_stream.cc:393
#5 0x0000555556b5c3d9 in webrtc::AudioMixerImpl::GetAudioFromSources(int) (this=0x7fffc40208f0, output_frequency=16000)
at ../../modules/audio_mixer/audio_mixer_impl.cc:205
#6 0x0000555556b5be4c in webrtc::AudioMixerImpl::Mix(unsigned long, webrtc::AudioFrame*)
(this=0x7fffc40208f0, number_of_channels=2, audio_frame_for_mixing=0x7fffc4025068) at ../../modules/audio_mixer/audio_mixer_impl.cc:175
#7 0x000055555699ef20 in webrtc::AudioTransportImpl::NeedMorePlayData(unsigned long, unsigned long, unsigned long, unsigned int, void*, unsigned long&, long*, long*)
(this=0x7fffc4024f60, nSamples=441, nBytesPerSample=4, nChannels=2, samplesPerSec=44100, audioSamples=0x7fff94001030, nSamplesOut=@0x7fffcd9f4348: 0, elapsed_time_ms=0x7fffcd9f4350, ntp_time_ms=0x7fffcd9f4358) at ../../audio/audio_transport_impl.cc:215
#8 0x00005555565b7e9b in webrtc::AudioDeviceBuffer::RequestPlayoutData(unsigned long) (this=0x7fffc4002c28, samples_per_channel=441)
at ../../modules/audio_device/audio_device_buffer.cc:302
#9 0x00005555565aa394 in webrtc::AudioDeviceLinuxPulse::PlayThreadProcess() (this=0x7fffc4004de0)
at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:2106
#10 0x000055555659e52d in webrtc::AudioDeviceLinuxPulse::<lambda()>::operator()(void) const (__closure=0x7fffc4007890)
at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:174
#11 0x00005555565ac068 in std::_Function_handler<void(), webrtc::AudioDeviceLinuxPulse::Init()::<lambda()> >::_M_invoke(const std::_Any_data &)
(__functor=...) at /usr/include/c++/9/bits/std_function.h:300
这里还想要捋出来 WebRTC 中音频发送和接收处理过程，各个部分的边界。WebRTC 的整体架构大概如下图所示：
WebRTC SDK 引擎
最下层的是媒体处理核心功能模块，上面是媒体引擎， 再上面是 PeerConnection 和 API，此外，WebRTC 还提供了大量的基础组件，如 socket，线程，锁 等等等。前面我们说明的音频发送和接收处理过程，除了最后 6. 音频数据包的发送处理 和 7. 音频数据包的接收处理 两个过程，其它都完全属于 PeerConnection。对于 6. 音频数据包的发送处理 过程，通过 BaseChannel 实现 MediaChannel::NetworkInterface 接口，并注册给 MediaChannel，将 PeerConnection 的发送能力和媒体处理核心功能组件连接起来。对于接收处理过程，边界则在 RtpTransport::DemuxPacket()，RtpTransport 是 PeerConnection 传输引擎的一部分，它通过 webrtc::RtpDemuxer 将收到的音频包送进核心的媒体处理功能中去。
网友评论