Markup in Multimedia

Related Multimedia Resources...
HyTime
SMIL

Markup in multimedia ...

Multimedia is the linking together of text, images, sounds, and actions into a nonsequential web of associations that permit the user to browse through related topics, regardless of the presented order of the topics. These links are often established both by the author of a hypertext document and by the user, depending on the intent of the hypertext document.

While developing SGML in the early 1980's, Charles Goldfarb, himself a competent jazz pianist became intrigued by the idea of using SGML for a standardized representation of music. He was joined by Steve Newcomb, then a professor of music at the University of Florida, and a new ISO project was initiated for a Standard Music Description Language (SMDL) based on SGML. One of the major challenges would be to devise ways to capture the temporal aspect of music and to be able to represent synchronization.

Myth has it that news of the project came to the attention of information technologists in the CIA who turned up at an ISO committee meeting one day claiming that SMDL was just what they needed in order to represent, well, certain matters of interest to them � which also required the ability to represent timing and synchronization abstractly. Could the committee please add certain details drop all references to music, which would sound "inappropriate" in the Department of Defense! To cut a long story short, SMDL was put on ice while its more general parts were developed separately, and the result was HyTime ([ISO 10744]), an ambitious standard that sought to provide ways of addressing and linking to any kind of information, anywhere in time and space.

HyTime was a major intellectual achievement and contained insights of immense value. But it also rapidly achieved the same reputation as Einstein's theory of relativity � of being totally impenetrable except to a handful of minds!

In order to help explain HyTime to the world, the GCA sponsored a project called "Conventions for the Application of HyTime" (CApH) whose goal was to come up with small subsets of HyTime, based on real use cases, that would be easier to understand and easier to implement than the full generality of HyTime itself. CaPH was driven principally by Newcomb and Michel Biezunski and one of the first real use cases they came up with was that of capturing the knowledge structures implicit in back-of-book indexes in order to able to automate merging and other processing of indexes. The original insights are due principally to Newcomb and the solution he and Biezunski devised was given the name "Topic Navigation Maps".

What we now call Topic Maps went through several years of gestation, with the work migrating in 1996 to ISO's SGML committee under the editorship of Martin Bryan and Biezunski. It was approved as an international standard in 1999 and published as [ISO 13250] in January 2000, by which time Newcomb was once more a driving force in its development. A year later, in March 2001, the "XML Topic Maps (XTM) Specification" ([Pepper 2001]) was published by an independent consortium called TopicMaps.Org, initiated by Newcomb and Biezunski and devoted to "developing the applicability of the topic map paradigm ... to the World Wide Web by leveraging the XML family of specifications."

The use of markup to coordinate multimedia events has been recognised for some time. The HyTime standard is a language for encoding the structure of hypermedia documents. It is an extension of SGML which encodes the structure of text documents. HyTime is defined as an SGML document architecture. As such, it extends SGML by defining how composites of SGML constructs can be built to represent hypermedia structure.

HyTime inherits from SGML the ability to define multiple document models, open and integrated documents, and document structure independently of document presentation. HyTime defines the following aspects of hypermedia document structure: addressing of document objects, relationships between document objects and numeric measurement of document objects.

HyTime is a standardized hypermedia structuring language for representing hypertext linking, temporal and spatial event scheduling, and synchronization. HyTime provides basic identification and addressing mechanisms and is independent of object data content notations, hyperlink types, processing and presentation functions, and other application semantics. Hyperlinks can be established to documents that conform to HyTime and those that do not, regardless of whether those documents can be modified. The full HyTime function supports "integrated open hypermedia" (IOH) - the "bibliographic model" of referencing that allows hyperlinks to anything, anywhere, at any time - but systems need support only the subset that is within their present capabilities.

The Synchronized Multimedia Integration Language (SMIL) is a recommendation for using XML to integrate a set of independent multimedia objects into a synchronized multimedia presentation. Using SMIL, an author can describe the temporal behaviour of the presentation, the layout of the presentation on a screen and associate hyperlinks with media objects.

SMIL enables simple authoring of interactive audiovisual presentations. SMIL is typically used for "rich media"/multimedia presentations which integrate streaming audio and video with images, text or any other media type. SMIL is an easy-to-learn HTML-like language, and many SMIL presentations are written using a simple text-editor.

SMIL is a simple language that can be easily crafted by hand. SMIL provides control of the semantically-related elements, properties, and attributes. SMIL brings interactive multimedia to the Web by providing a standardized format for scripting multimedia events encoded in XML. SMIL is actually a family of standards for different environments. SMIL is supported in RealPlayer and Quicktime as well as having its own dedicated players.