Editing Documentation/Maemo 5 Developer Guide/Using Multimedia Components/Introduction
Warning: You are not logged in.
Your IP address will be recorded in this page's edit history.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
| Latest revision | Your text | ||
| Line 1: | Line 1: | ||
| - | + | =Introduction= | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
Maemo offers lots of possibilities for multimedia applications. Maemo was built with the Internet Tablet devices in mind - hardware in this form factor provides a big high resolution touch screen, audio input and output, video and image capturing through the camera, fast network connections for streaming and a digital signal processor for efficient audio and video manipulation. | Maemo offers lots of possibilities for multimedia applications. Maemo was built with the Internet Tablet devices in mind - hardware in this form factor provides a big high resolution touch screen, audio input and output, video and image capturing through the camera, fast network connections for streaming and a digital signal processor for efficient audio and video manipulation. | ||
For developers, there are open programming interfaces to make use of these features apart from directly programming the DSP. The preferred way of doing video and audio programming for Maemo is using the [http://gstreamer.freedesktop.org/ GStreamer framework]. There are several libraries available for manipulating images. Interactive multimedia for applications like games can be done using the [http://www.libsdl.org/ SDL framework]. | For developers, there are open programming interfaces to make use of these features apart from directly programming the DSP. The preferred way of doing video and audio programming for Maemo is using the [http://gstreamer.freedesktop.org/ GStreamer framework]. There are several libraries available for manipulating images. Interactive multimedia for applications like games can be done using the [http://www.libsdl.org/ SDL framework]. | ||
| + | |||
| + | |||
==GStreamer - Multimedia Framework== | ==GStreamer - Multimedia Framework== | ||
| - | |||
The main multimedia framework in Maemo is [http://gstreamer.freedesktop.org/ GStreamer]. It is based on the concept of pipelines, which are made of multiple elements. Elements themselves can be practically anything that do something with a data stream. | The main multimedia framework in Maemo is [http://gstreamer.freedesktop.org/ GStreamer]. It is based on the concept of pipelines, which are made of multiple elements. Elements themselves can be practically anything that do something with a data stream. | ||
| - | Maemo includes a variety of these elements to support audio and video effects, encoding and decoding and interfacing with the device's hardware. GStreamer also provides high level elements for specific use cases, like | + | Maemo includes a variety of these elements to support audio and video effects, encoding and decoding and interfacing with the device's hardware. GStreamer also provides high level elements for specific use cases, like playbin2 for playback and camerabin for image and video capture. Using this approach, a developer does not have to bother with details like low-level hardware management, audio and video compression schemes, etc. If the elements included with the OS are not enough, more can be either compiled for Maemo, or new ones can be developed. |
Developer documentation for GStreamer can be found at the [http://gstreamer.freedesktop.org project's website]. | Developer documentation for GStreamer can be found at the [http://gstreamer.freedesktop.org project's website]. | ||
| + | |||
==MAFW== | ==MAFW== | ||
| + | Media Application Framework (MAFW) is a middleware layer for all mediaplayer applications. It provides generic APIs for browsing and searching multimedia content (music, video, images) from various services (e.g. local filesystem, UPnP, etc.) and an extensible plugin architecture which allows providing new content sources via new service plugins. In addition to content metadata access, MAFW provides also generic APIs for playback of media (music, video, images) via various rendering services (e.g. gstreamer, UPnP, etc.). Also new rendering services can be added via a plugin mechanism. MAFW allows 3rd party developers flexible ways to implement new content services support for the Maemo platform. | ||
| + | |||
| - | |||
==Stream Encoding and Decoding== | ==Stream Encoding and Decoding== | ||
| - | + | Maemo devices include a wide arsenal of video and audio codecs, which enable the Maemo applications to read and write nearly all commonly used video and audio formats. These are supported also by the GStreamer framework. For many purposes the developer does not even have to specify the used codecs implicitly, since GStreamer automatically detects the format and the codec to use. The [http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-playbin.html playbin] GStreamer base plug-in provides a convenient abstraction layer for all audio and video content. | |
| - | Maemo devices include a wide arsenal of video and audio codecs, which enable the Maemo applications to read and write nearly all commonly used video and audio formats. These are supported also by the GStreamer framework. For many purposes the developer does not even have to specify the used codecs implicitly, since GStreamer automatically detects the format and the codec to use. The [http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins- | + | |
Due to non-technical reasons, most of the codecs are not distributed with the SDK. This is good to keep in mind when developing applications relying on such features. | Due to non-technical reasons, most of the codecs are not distributed with the SDK. This is good to keep in mind when developing applications relying on such features. | ||
| + | |||
==Digital Signal Processor== | ==Digital Signal Processor== | ||
| - | |||
Inside the Maemo device, there is a dedicated digital signal processor (DSP). Its design is optimized for tasks like stream encoding/decoding and audio effects. Maemo has high-level programming support for the DSP in the form of GStreamer elements, which can decode most of the supported file formats. By using the DSP, the computing load is also taken off from the main processor, greatly enhancing system performance and responsiveness. | Inside the Maemo device, there is a dedicated digital signal processor (DSP). Its design is optimized for tasks like stream encoding/decoding and audio effects. Maemo has high-level programming support for the DSP in the form of GStreamer elements, which can decode most of the supported file formats. By using the DSP, the computing load is also taken off from the main processor, greatly enhancing system performance and responsiveness. | ||
===Audio=== | ===Audio=== | ||
| - | |||
For audio programming, Maemo has several main APIs: GStreamer and PulseAudio. Usually system sounds, such as sounds for notifying user of an event, e.g. battery low, are played through PulseAudio. One can use libcanbera if a simple API is needed. More sophisticated operations, e.g. playing music files or recording audio, should be generally performed using GStreamer, which a much more flexible API. Most of Maemo's computationally intensive GStreamer elements are implemented using the device's DSP which greatly enhances their performance. | For audio programming, Maemo has several main APIs: GStreamer and PulseAudio. Usually system sounds, such as sounds for notifying user of an event, e.g. battery low, are played through PulseAudio. One can use libcanbera if a simple API is needed. More sophisticated operations, e.g. playing music files or recording audio, should be generally performed using GStreamer, which a much more flexible API. Most of Maemo's computationally intensive GStreamer elements are implemented using the device's DSP which greatly enhances their performance. | ||
| Line 40: | Line 36: | ||
==Video== | ==Video== | ||
| - | |||
Although the framework hides much of the implementation and hardware details, it is good for a developer to know what happens beneath the interfaces. Video capturing is performed via the Linux kernel's Video4Linux2 API, and graphics are displayed using the X Window System, namely the XVideo extension. Practically all GNU/Linux applications rely on these components for video tasks, so porting existing applications should be quite effortless, and support easy to find. | Although the framework hides much of the implementation and hardware details, it is good for a developer to know what happens beneath the interfaces. Video capturing is performed via the Linux kernel's Video4Linux2 API, and graphics are displayed using the X Window System, namely the XVideo extension. Practically all GNU/Linux applications rely on these components for video tasks, so porting existing applications should be quite effortless, and support easy to find. | ||
| - | Hands-on instructions for using capture and output features are given in | + | Hands-on instructions for using capture and output features are given in section [localhost#sec:camera_api_usage 8.3]. |
| - | |||
| + | ==Graphics and Images== | ||
Maemo includes several libraries for working with images and graphics: | Maemo includes several libraries for working with images and graphics: | ||
| Line 54: | Line 49: | ||
Visit [http://cairographics.org/ Cairo website] for guides, tutorials and references. GDK's and GdkPixbuf's documentation can be found at [http://www.gtk.org/ GTK+ project's website]. | Visit [http://cairographics.org/ Cairo website] for guides, tutorials and references. GDK's and GdkPixbuf's documentation can be found at [http://www.gtk.org/ GTK+ project's website]. | ||
| + | |||
==Games== | ==Games== | ||
| - | + | For game developers, maemo offers a common start-up and settings screen, and a framework called ''osso-games-startup''. Using this library, game developers and porters can reduce the common start-up screen coding, and concentrate on the real game programming. Osso-games-startup's operating system communication features also offer an interface to make games behave correctly. It eases, for example, the management of full screen mode and exceptional situations, such as battery low notification, in a unified and user friendly manner. See section [localhost#sec:games_start-up_screen_usage 8.6] for more in-depth description and API usage. | |
| - | For game developers, maemo offers a common start-up and settings screen, and a framework called ''osso-games-startup''. Using this library, game developers and porters can reduce the common start-up screen coding, and concentrate on the real game programming. Osso-games-startup's operating system communication features also offer an interface to make games behave correctly. It eases, for example, the management of full screen mode and exceptional situations, such as battery low notification, in a unified and user friendly manner. See | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
Learn more about Contributing to the wiki.