pidgin/nest

Migrates Voice and Video from wiki

2019-03-17, Jason
424ebf71e852
Parents 11ce48aadf3d
Children 2e12c8034c19
Migrates Voice and Video from wiki
2 files changed, 173 insertions(+), 0 deletions(-)
  • +90 -0
    hugo/content/drafts/voice-and-video-API.md
  • +83 -0
    hugo/content/drafts/voice-and-video.md
    • --- /dev/null Thu Jan 01 00:00:00 1970 +0000
    +++ b/hugo/content/drafts/voice-and-video-API.md Sun Mar 17 23:11:17 2019 +0000
    @@ -0,0 +1,90 @@
    +---
    +title: "Voice and Video API"
    +date: 2019-03-16T20:15:02Z
    +replaces: [
    + "developer.pidgin.im/wiki/vvAPI",
    +]
    +---
    +
    +For functions that could be per conference, session, participant, or stream, they would have a session and a participant parameter. A NULL for either of these would be a wildcard.
    +
    +### PurpleMediaManager?
    +
    +#### Functions
    +
    +```
    +*_initiate # Creates a PurpleMedia instance. Probably have a flag as to whether it's the initiator or responder
    +```
    +
    +#### Signals
    +
    +```
    +init-media # Same as the current signal. Is triggered when *_initiate is called
    +init-video-src # These are all triggered when the corresponding *_get_\* functions are called
    +init-video-sink # The one problem I've thought of with them is that this way it wouldn't
    +init-audio-src # be very easy to have multiple srcs or sinks of the same type.
    +init-audio-sink
    +```
    +
    +#### Get
    +
    +```
    +*_get # Returns a global instance of the PurpleMediaManager
    +
    +*_get_pipeline # Returns a global pipeline to be used for all PurpleMedia instances.
    +
    +*_get_video_src # These all create their respective types if they don't exist.
    +*_get_video_sink # Otherwise they return a src/sink-pad for the existing src/sink.
    +*_get_audio_src
    +*_get_audio_sink
    +
    +# The next three functions will probably have a type parameter (audio or video, src or sink)
    +*_get_plugins # Return a list of audio/video GStreamer plugins (eg. v4lsrc, alsasrc)
    +*_get_devices # Return a list of devices for a specific plugin
    +*_get_devices_autodetect # Eventually, return a list of devices detected for all supported plugins
    +```
    +
    +### PurpleMedia?
    +
    +#### Functions
    +
    +All of these functions will have parameters for session_id and participant_id NULL for either of these would indicate a wildcard and do this operation for all matching streams
    +
    +```
    +*_add_stream # Adds a stream/session/participant to the conference, automatically creates participants and FsSessions as necessary
    +*_add_candidate # Adds a single candidate to the stream (May not be necessary. Could alternately add a list)
    +*_set_candidates # Sets a list of candidates for the stream
    +*_set_codecs # Sets a list of codecs for the stream
    +*_set_direction # Changes the direction of a stream/session/participant (could use *_add_stream to accomplish this)
    +
    +*_accept # Accept a session (User clicked the accept button)
    +
    +# The following two could probably be merged into one function
    +*_mute # Mutes an audio stream/session/participant (sending and/or receiving?)
    +*_pause # Pauses a video stream/session/participant (sending and/or receiving?)
    +
    +*_end # Ends a stream/session/participant/conference
    +```
    +
    +#### Signals
    +
    +These signals also apply to the wildcard note above
    +
    +```
    +ready # Waits for both codecs-ready to be TRUE and candidates-prepared to have fired (possibly also wait for the user to click accept if they are the responder otherwise we'll probably need an "accepted" signal)
    +state-changed # Has an enum state (containing such states as connected, end, and ice-specific values) Signals for stream/session/participant/conference
    +```
    +
    +More signals may be necessary for different protocols. Jingle ice-udp may also need a new-candidate and/or new-candidate-pair and a codecs-changed equivalent
    +
    +#### Get
    +
    +I'm not certain all of these will be necessary
    +
    +```
    +*_codecs # gets codecs per sessions
    +*_candidates # gets candidates per stream
    +*_streams # gets stream_ids by session or by participant
    +*_sessions # gets session_ids in the conference
    +*_participants # gets participant names by conference or by session
    +```
    --- /dev/null Thu Jan 01 00:00:00 1970 +0000
    +++ b/hugo/content/drafts/voice-and-video.md Sun Mar 17 23:11:17 2019 +0000
    @@ -0,0 +1,83 @@
    +---
    +title: "Voice and Video"
    +date: 2019-03-16T16:58:31Z
    +replaces: [
    + "developer.pidgin.im/wiki/vv",
    +]
    +---
    +
    +## Voice and Video
    +
    +Voice and video in libPurple is being worked on using [​Farstream](http://www.freedesktop.org/wiki/Software/Farstream/) (formerly Farsight). There are a few parts to this implementation. Firstly, basic API support needs to be added to libPurple. Then Pidgin and Finch need to be able to initiate, participate in, and terminate the voice and video sessions. After these two requirements are met, each protocol will require both the signaling to be implemented (such as SIP or Jingle) and the proper Farstream plugins.
    +
    +Farstream has rawudp, iceudp, and multicast transmitter plugins and an MSN webcam plugin. Basic voice and video support for libPurple, Pidgin, and Finch (voice support only) have been added to the im.pidgin.pidgin development branch. XMPP/GTalk/Gmail is working, but not complete. None of the other protocols have had much work done.
    +
    +### Protocol Specific Information
    +
    +#### XMPP/GTalk
    +
    +XMPP, GTalk, and Gmail use RTP. XMPP uses Jingle. GTalk and Gmail use their own flavor of Jingle. XMPP uses rawudp and XMPP, GTalk, and Gmail use the libnice Farstream transmitter plugin.
    +
    +Voice and video support for XMPP/GTalk was worked on for a [Summer of Code project](/wiki/GSoC2008/VoiceAndVideo)
    +
    +#### MSN
    +
    +MSN has several different voice and video types:
    +
    +- MSN Audio Conference (WLM 2009)
    +- MSN Video Conference (WLM 2009. Includes audio. Allows the h263 codec)
    +- MSN Audio Conference (pre-WLM 2009. Sometimes works with WLM2009)
    +- MSN Video Conference (pre-WLM 2009. Sometimes works with WLM2009. Includes audio. Only uses the wmv3 codec for which there is no open source encoder/GStreamer element)
    +- MSN Webcam
    +
    +A Farsight2 plugin for MSN Webcam has been released.
    +
    +[​http://imfreedom.org/wiki/MSN:AV](http://imfreedom.org/wiki/MSN:AV) has more information about the protocol.
    +
    +#### Oscar (AIM, iChat, ICQ)
    +
    +All Oscar protocols use RTP and SIP.
    +
    +#### Yahoo
    +
    +Yahoo voice uses straight SIP. It binds to the SIP server upon login and sends INVITEs as necessary.
    +
    +Yahoo webcam uses the JPEG2000 codec with proprietary transport and signalling. The old Farsight Yahoo plugin will need to be ported to the new Farstream API.
    +
    +#### Sametime
    +
    +Sametime seems to support at least voice chat [​http://publib.boulder.ibm.com/infocenter/sametime/v8r0/index.jsp?topic=/com.ibm.help.sametime.standard.doc/st_adm_avserv_aboutvoicechat_r.html](http://publib.boulder.ibm.com/infocenter/sametime/v8r0/index.jsp?topic=/com.ibm.help.sametime.standard.doc/st_adm_avserv_aboutvoicechat_r.html)
    +
    +#### Gadu-gadu
    +
    +This is a placeholder for information regarding GG voice An announcement that might be of interest: [​http://www.finanznachrichten.de/nachrichten-2008-10/12104971-global-ip-solutions-ab-gadu-gadu-launches-new-version-of-instant-messenger-with-excellent-voice-powered-by-gips-004.htm](http://www.finanznachrichten.de/nachrichten-2008-10/12104971-global-ip-solutions-ab-gadu-gadu-launches-new-version-of-instant-messenger-with-excellent-voice-powered-by-gips-004.htm)
    +
    +### Building with Voice and Video
    +
    +Please be familiar with compiling and installing code from source. Please also submit any bugs you find to Trac ([TipsForBugReports](/wiki/TipsForBugReports)).
    +
    +Make sure you are using the latest release of Pidgin.
    +
    +Farsight 2 0.0.9 and libNice 0.0.7 are the minimum versions required, but the newest versions you can get are best. Make sure to check Farsight2's README and make sure you have all of the dependencies for it. gst-plugins-base is also required for the gstreamer-interfaces-0.10 interface. Some distros call it libgstreamer-plugins-base0.10-dev. If everything in order, the configure script should output this line:
    +
    +> Build with voice and video.... yes
    +
    +For help with building, see [Installing Pidgin](/wiki/Installing%20Pidgin#Compiling). Please carefully read through that section and any extra information for the distribution you're using.
    +
    +#### Codec Configuration
    +
    +If a certain codec isn't working for you, you need to create an "fs-codec.conf" file in your purple config directory. See this for the specification: [​fs-codec-list-from-keyfile](http://farsight.freedesktop.org/apidoc/farsight2/farsight2-libs-FsCodec.html#fs-codec-list-from-keyfile)
    +
    +So, for example, if you want to disable the MPV codec you'd put this in the fs-codec.conf file:
    +
    +```
    +[video/mpv]
    +id=-1
    +```
    +
    +### Links
    +
    +- [​http://www.freedesktop.org/wiki/Software/Farstream/](http://www.freedesktop.org/wiki/Software/Farstream/)
    +- [​http://code.google.com/apis/talk/libjingle/index.html](http://code.google.com/apis/talk/libjingle/index.html)
    +- [​http://trac.adium.im/wiki/VoiceAndVideo](http://trac.adium.im/wiki/VoiceAndVideo)
    +- [​http://trac.adium.im/wiki/PidginVV](http://trac.adium.im/wiki/PidginVV)