美文网首页
Linphone-mediastreamer2

Linphone-mediastreamer2

作者: Flannery | 来源:发表于2016-07-19 11:10 被阅读0次
    Linphone学习目标

    Description
    Mediastreamer2 is a powerful and lightweight streaming engine specialized for voice/video telephony applications.
    This open source library is responsible for all the receiving and sending of multimedia streams in Linphone, including voice/video capture, encoding and decoding, and rendering.


    Mediastreamer2 in Linphone architecture

    schema-mediastreamer2

    Features


    Features
    Capture and playback from various platform dependent sound architectures (ALSA, PulseAudio, AudioUnit, AudioQueue, WaveApi, WASAPI, Android AudioTrack/AudioRecord, Android OpenSLES)
    Send and receive RTP streams
    Encode and decode the following audio formats: OPUS, speex, G711, GSM, iLBC, AMR, AMR-WB, G722, SILK, G729
    Encode and decode the following video formats: VP8 (WebM), H263, H263-1998, MPEG4, theora and H264 (thanks to a plugin based on OpenH264), with resolutions up to 1080P
    RTP/AVPF support: RTCP control messages for video error recovery: PLI, SLI, RPSI, FIR.
    Audio conferencing
    Supports SRTP and zRTP (encryption of voice and video)
    Supports any webcam, based on OS's camera API: quicktime, directshow, video4linux, android.camera
    Acoustic echo cancellation using the speex echo canceler or webrtc AECm.
    Read and write from/to a wav file
    Optimized rendering of YUV pictures, using openGL, DrawDib, X11/Xv
    Dual tones generation
    Custom tone detector
    Audio parametric equalizer
    Volume control, automatic gain control
    ICE for optimized NAT traversal (RFC5245) to allow peer to peer audio & video connections without media relay server
    Adaptive bit rate control algorithm: for automatic adaption of encoder bit rate based on received RTCP feedback.
    Can use plugins: to add new codecs, new sound input/output backend...


    Portability
    Google Android >= 2.2
    iOS >= 6
    Windows XP, Vista, 7 and 8
    Mac OS X 10.7 to 10.10
    Linux Debian 6/7, Centos 6/7
    Embedded Linux: ARM
    BlackBerry OS10
    Windows Phone 8 and 8.1


    Design and principles
    Each processing entity is contained within a MSFilter object. MSFilter(s) have inputs and/or outputs that can be used to connect from and to other MSFilters.
    A trivial example to understand:
    MSRtpRecv is a MSFilter that receives RTP packets from the network, unpacketize them and post them on its only output.
    MSSpeexDec is a MSFilter that takes everything on its input assuming these are speex encoded packets, and decodes them and put the result on its output.
    MSFileRec is a MSFilter that takes everything on its input and write it to wav file (assuming the input is 16bit linear pcm).

    MSFilters can be connected together to become filter chain. If we assemble the three above examples, we obtain a processing chain that receives RTP packet, decode them and write the uncompressed result into a wav file.MSRtpRecv --> MSSpeexDec --> MSFileRec
    The execution of the media processing work is scheduled by a MSTicker object, a thread that wakes up every 10 ms to process data in all the MSFilter chains it manages. Several MSTicker can be used simultaneously, for example one for audio filters, one for video filters, or one on each processor of the machine where it runs.


    Media protocols
    RTP: A Transport Protocol for Real-Time Applications, RFC 35505. RTP Data Transfer Protocol
    6.4 Sender and Receiver Reports
    6.5 SDES: Source Description RTCP Packet
    6.6 BYE: Goodbye RTCP Packet
    6.7 APP: Application-Defined RTCP Packet

    RTP Profile for Audio and Video Conference with Minimal Control, RFC 3551
    Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF), RFC 4585
    Symmetric RTP / RTP Control Protocol (RTCP), RFC 4961
    Session Traversal Utilities for NAT (STUN), RFC 5389 (Basic procedures)
    Secure Real Time Transport Protocol (SRTP, RFC 3711)
    ZRTP, RFC 6189
    ICE, RFC 5245 & RFC 6336
    RTP Payload for Text conversation, RFC 4103


    Audio protocols
    RTP Payload Format for the Speex Codec, RFC 5574
    Real-Time Transport Protocol (RTP) Payload Format and File Storage Format for the Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband (AMR-WB) Audio Codecsonly RTP octet-align=1 mode, without interleaving, crc, single channel

    RTP Payload for DTMF Digits, Telephony Tones, and Telephony Signals, RFC 4733RTP Payload Format for Named Telephone Events

    RTP Payload Format and File Storage Format for SILK Speech and Audio Codec

    Video protocols
    XML Schema for Media Control, RFC 5168Sending and processing of picture fast update command in SIP INFO

    RTP Payload Format for MPEG-4 Audio/Visual Streams, RFC 30163. RTP Packetization of MPEG-4 Visual bitstream

    RTP Payload Format for ITU-T Rec. H.263 Video, RFC 4629
    RTP Payload Format for H.264 Video, RFC 3984
    RTP Payload Format for VP8 Video


    Documentation


    Mediastreamer2 is documented using doxygen. You can browse the API documentation here.

    mediastreamer2 Documentation
    Project Website: http://www.mediastreamer2.com

    相关文章

      网友评论

          本文标题:Linphone-mediastreamer2

          本文链接:https://www.haomeiwen.com/subject/rwpyjttx.html