The Convergence of Computers and Video = Digital Fusion Impact of Multimedia Digital Fusion Hypermedia Model of Multimedia   Digital fusion - multimedia* - desktop media - is described as the convergence of computing, television, audio, printing and telecommunications. Each of the components has been significant. Bringing them together results in the whole having greater impact than each individual part and is one of the industry's most significant developments. The convergence of digital technologies and their use will impact the future of teleconferencing, distance education and business (Arnett & Greenberg, 1989). (*Not to be confused with the multimedia shows of the 70s-80s which were slide and film live presentations.) Multimedia systems are those that are able to control some or all of the tasks associated with creation, development, production and post production via a single easy-to-use universal graphic console. Multimedia promises to make desktop video as significant in the 90s as desktop publishing was in the 80s. Spurred by personal computing, desktop publishing brought to millions of users the ability to create high-quality printed materials at low cost. The result has been a phenomenal rise in the amount of printed matter which is an irony in an age of electronic information. Impact of Multimedia Computing The fusion of these technologies will radically change how communications occur in most organizations. The demand for computers, high-capacity data communications and storage will soar as publishing of electronic documents rises. Fiber optic networks and optical storage systems are the interstate highways and warehouses of the information age. Fiber optic can transmit voice, video and data by light wave signals. With a fiber optic infrastructure, video can be exported from a studio or imported to the home. Other technologies are being explored which will also provide the bandwidth required to carry voice, data and video. These include Asynchronous Transfer Mode (ATM) and integrated services digital network(ISDN). Computer users will have greater control over the information they receive from news, information and entertainment services. Interactive educational and reference systems will offer students new ways to learn and research but at a high market price, unless industry and government work put the technology into schools and libraries. Audiovisual, graphic production and communications skills will become increasingly valuable in education, business and social circumstances, shifting power structures away from "left brain" (details) achievers toward a middle ground. Nicholas Negroponte, MIT Media Lab director, says that to regard some students as "learning disabled" may disappear in favor of regarding their schools as "teaching disabled" because they do not have interactive audio-visual educational systems. More information will be available about institutions and individuals. The ability to defend the right to privacy will continue to erode. Accelerating information overload will drive the demand for digital storage of more kinds and amounts of information, as well as expert systems and related technologies to sort through the variety of sources. The obsolescence of intellectual property rights will continue as an even greater problem. Digital communication networks that treat information as a commodity will rise; individuals and corporations will make information and services available in a free market. Electronic mail is close to becoming a standard mode of communication rather than "snail mail." Digital Fusion Digital fusion has given computers the ability to communicate and TV the ability to manage information. Importantly, digital fusion will give control to the non-specialist. Just as desktop publishing opened publishing to ordinary people, desktop video will also open video to ordinary people. This includes productions done by elementary school children and university professors. The future is being shaped by new hardware and software that apply the personal computer's low-cost information management capabilities to television's uncanny ability to motivate and communicate. TV is not just video images. It is a complete commercial mass communications form, which includes sound, music, computer graphics and animation, as well as highly developed styles of editing and production. The merger of computer and video technologies spells an economic challenge at the core of the U.S.'s serious trade conflicts with Japan according to Arnett and Greenberger (1989). The predominantly U.S. based personal computer industry will be forced to compete or cooperate with the largely Japanese consumer electronics industry as the two find themselves pushing toward similar goals. The technological merger feeds upon itself. Driving the demand is the need to deal with the proliferation of information flowing from the same technologies that are being joined: computers, TV, printers and telephones (Arnett & Greenberger,1989). TV, which makes us laugh and cry, brings feeling to the coldness of data processing. The computer, which helps us manage information, encourages us to stop and think in the midst of - and as an antidote to - television's frantic and frequently captivating stimulation. Using TV was a matter of choosing one of a few channels. Information overload has made watching TV more complex, with the addition of cable channels, tapes, camcorders, remote controls and programmable VCRs. Yet, TV remains largely superficial and unable to offer information on demand. By joining TV and computers, we combine the best aspects of each technology. The result is a powerful communications and information system that joins TV's ability to introduce and highlight a subject with the computer's ability to provide in-depth information tailored to needs. The computer changes existing media by helping one find, store, search and re-use many kinds of information. "Interactivity" is the term to describe this ability, or need, to control what is happening. Two-way communications have the highest level of interactivity, whether the communication is with a person or with a machine. A prime objective of multimedia computing is for television's superficiality to become more appropriate and function so that it would serve as an introduction, or motivator, to the detailed information that the computer makes available through its ability to store, search and retrieve. (Arnett & Greenberger, 1989). It is important to understand what motivates the way that each form of media presents information. Printed text (newspapers, magazines, books) allow you quickly scan to see which items you want to read. Newspapers use the inverted pyramid where the most important information is at the top of the story or at the front of the issue; readers aren't expected to read everything, but read as much as they like without worrying about missing anything. This encourages readers to stop and think about whether they want to learn more about the subject or go on. Television's organizational style encourages one to watch everything, without scanning or considering whether the appetite for it has been satiated. It does this through the use of levels of emotional and sensory stimulation. The focus on feeling and drama is not bad; but it is an incomplete perspective. Analysis and action are discouraged by TV's goal to be exclusive unto itself (Arnett & Greenberger, 1989). Computers tend to present narrowly focused information in the form of spreadsheets, databases or words. It's hard to establish a relationship between different kinds of computer information because the data format of each is likely to be different Hypermedia Hypermedia, refers to software that can accept information in a variety of formats - text, images, graphics,sounds and video, and to link related things together interactively to create a single multimedia presentation. Hypermedia became the method that was used to create the Internet's World Wide Web. Using code called HTML (Hypertext Markup Language) and another programming language called Java, hypermedia can branch to a motion video presentation within a window on the computer screen, to text, or any combination of information that moves, rotates, or transforms itself through inventive software programming. Interactive documents represent a fundamental shift in the way computers are used because interactivity demands that the computer become the delivery medium. An interactive document requires the user's active participation or it stops. Like television, a multimedia document can present an overview with sensory and emotional information; like a newspaper, it lets one browse and choose how much information is wanted; like a computer file, it will allow the re-use of parts and add others to create new documents; and it allows and facilitates communication with others. These capabilities lend themselves to a four-layer structure for multimedia information products. They are: an audiovisual surface, an information navigation system, information content, and creative tools (see chart). For a student, the four layers would correspond to hearing lectures, research, reading and writing. For a shopper, they correspond to watching advertising, shopping, purchasing and using products (Arnett & Greenberger, 1989). As the Web became an important part of information dissemination for people, companies and other organizations, new software programs were developed that divided the computer screen into frames. Using a frame, a content section could always be visible, along with other hot buttons that could take the user to other places on the site or to related sites. Interestingly, while frames were meant to make the use of a web site easier, it was found that users needed a great deal of sophistication to follow the information to a destination that they wanted rather than just "surfing" the Web. In particular, users who were not reading at adult levels or where English was not their first language, found the frames to be more confusing than helpful. When a higher level of educational structure was provided at the sites which provided literacy training, the users were able to easily use Internet for their lessons. The Audiovisual Surface Audiovisual introductions, using video or graphics, are effective because they make few assumptions about the viewer's interests and motivations. An audio-visual introduction helps viewers find something in which they are interested, or motivates them to become interested. For example,the first thing that happens on an ABC News videodisc is that Peter Jennings appears and explains what the disc is about and how it works. After the introduction, the user decides where to go within the multimedia document. This could range from a single database or text file to all of the world's databases accessible by telephone. The challenge here is to determine which to search. The computer's storage and programmability turn indexes into active, two-way links among parts of a document or even between different documents. A reference to "Hamlet" in a book about politics can trigger on demand a link directly to the text of "Hamlet", and back. Powerful hypermedia software can create these kinds of contextual links, but there is a tremendous amount of manual labor involved. Global indexing and linking requires standards that are only beginning to be discussed by the international standards organization. Content ABC videodiscs which are viewed with a Macintosh computer and HyperCard, were developed to give teachers a compelling way to present information about recent events ahead of new textbook editions. They do not replace text; computer-stored text supplements the video. Television's motivational power draws the students into reading. Users play the disc interactively, using computers that provide the ability to access simultaneously the graphic or textual information. ABC produced a videodisc on the California earthquake in two weeks and sold it six weeks after the quake; it has produced discs on the 1988 presidential campaign and the Israeli-Palestinian conflict. More exciting is the use of videodiscs by students to create video reports which will not require skill in computers or video. Rich material is available from ABC and WGBH's interactive NOVA series. DVD - digital video disc - is the newest way to provide video. DVD discs are significantly smaller than the LP format of the older videodisc. CD-ROMS are now read and write for costs that are entering consumer and school costs. For many students, the transformation from being a passive television consumer to being a creator of a viewable production, will be a watershed event. Students effectively become "programmers" in the computer and television meanings of the term and are likely to become more discerning and critical television viewers, just as improved writers become better readers (Arnett & Greenberger, 1989). Access to content on the Internet has increased the speed with which students can learn about new events. It has opened the door to the world for everyone. The graphic user interface has an importance that is often overlooked in that it provides a kinesthetic component. The mouse adds movement to the computer interface. Using the mouse to put something on the screen, helps you remember it, just as it is easier to remember a phone number if you punch in the numbers with your fingers. The kinesthetic component involves motor-skills to move the mouse and type words, linguistic skills to create the words, visual graphic skills (seeing the graphic element on the screen), and visual linguistic skills to read the words on the screen (Arnett & Greenberger, 1989). Research is showing that the more areas of the brain that are involved in learning, the easier it is to retrieve the information in the future. Creative Tools This is the most active layer. There are two kinds of production: one is production that happens entirely inside the computer, such as writing, creating graphics or animation; the second involves capturing real-world events, such as sounds and images, and storing them on computer disks. In this step, the materials that have been gathered may be synthesized, new material may be created and a new production results. To combine source materials software tools are needed. This software is likely to be increasingly "object-oriented," because it allows users to deal with information realistically (Arnett & Greenberger, 1989). The difference between a computer paint program and a draw program is object orientation. When the user paints with the computer, the only information stored is whether a dot (pixel) is, black, white or a color. To change the pixel, it has to be erased and re-painted. In an object-oriented program, the computer stores the mathematical description of the object (such as a circle or square); changing the position or size is done without erasing the old one. Apple's HyperCard uses on-screen buttons and text fields that are programmed by the stack's author in an endless number of situations. (A stack is a completed file that runs under HyperCard). Graphics can be imported. HyperCard has a degree of object orientation that lets users copy function buttons from one HyperCard stack to another, yet retain their functions. It allows ordinary users to create customized programs when they would not have considered programming with structured languages and gives them control that previously was beyond their reach. Reusability means that users do not have to reinvent the wheel for each program. A typical stack is like a pile of index cards that can be linked together for quick reference. The top card might show a television; clicking on the fine-tune button takes the user rapidly (with visual and sound effects) to another card that explains fine-tuning. Clicking on a return arrow sends the user back to the main card. The next iteration of HyperCard was the use of the same conventions as an interface for the Internet. First came Mosaic which made all of us realize how useful the Internet could be when the Interface was user-friendly and masked the technology so that the content became easily accessible and easy to acquire. Next came NetScape and Navigator. The use of Internet has become even more useful with multitasking programs that allow Internet to be accessed while the user is working in a word processing application. Critics of multimedia argue that the average user will never be able to create quality productions. This argument overlooks present and future computer capabilities, as well as acceptance of lowered quality levels. Given the content of personal communication, the demand for production quality goes down (Arnett& Greenberger, 1989). Example: home video of children; the video may be jumpy, but the viewer is so interested in the content that he or she does not focus on the production values. There will always be a need and desire for higher quality provided by professional applications and hypermedia offers professionals faster, easier and less expensive ways of doing production. Data Storage Information is stored on magnetic and optical discs and magnetic tape. Some forms of storage are removable such as floppy disks, CD-ROM discs and DVD discs. Large capacity drives such as Zip and Jazz drives carry enormous amounts of information and have taken the place of other storage media for some jobs. Some kinds of removable discs can be written onto. CD-ROM used to fit this criteria and because of that it was used extensively for publishing. Since most computers, including laptaps, are now built with CD-ROM drives, and some are being built with DVD drives, it seems likely that this type of media will continue to be at the center of the market. Removable media are useful for distribution and storage. Non-removable (or fixed- storage systems, such as hard disk drives), are useful for local storage of data. Multimedia systems demand high-capacity data storage because of the huge amount of data necessary for high-quality color image, animation, audio, video, and the need to keep it in a digital electronic format to preserve interactivity. Using the most advanced video compression systems, it's possible to store just over an hour of digital video on the highest capacity, widely available kind of disk, CD-ROM. Technology presented us with DVD so now we can play a full-length movie. Computer drives are now large enough to store libraries of images and other information which should support most multimedia productions. What is really needed is the ability to store libraries of images and other information. The average computer's disk drive cannot store more than a few minutes of digital video data. Four-layer Model of Multimedia Products and Systems Every multimedia product doesn't include all four layers. For multimedia computing to achieve its greatest potential, products will have to be designed to fit into this framework, to address basic market needs (Arnett & Greenberg, 1989). I. Audiovisual Surface - Viewing, Browsing Offers overviews and motivation Less Interactive - Viewing Examples: Audiovisual introductions Still graphic presentations Text overviews Hardware Videotape players Video projectors Video encoders scan converters monitors Software Multimedia/video post-production systems Animation Digital audio Modeling More Interactive - Browsing Examples Video "samplers" Animation Graphics Hardware Videodisc players Video digitizers Software Hypermedia Menuing systems Synthetic actors and voices II. Navigation - Hunting, Grazing Information navigation to find additional materials Less interactive - Grazing Examples Hypertext applications Hypermedia applications tables of contents Software Hypermedia Expert systems Contextual linkers Recognition systems (voice, image, etc.) More interactive - Hunting Examples Query by example Key word search Index search Software Database query generators Indexers Authoring Systems III. Information = Reading, Viewing, Copying, Storing Content - the actual information that is stored in the system Examples Electronic mail messages On-line libraries of text graphics, video Non-reference materials Encyclopedias Databases Reference materials of all kinds Hardware Optical disk drives Magnetic drives Other storage systems Paper Videotape IV. Creative and Publishing Tools - Using Creative and publishing tools that are used to manipulate existing information or create new material for distribution. Production Everything above that creates new material Post-Production Everything above that helps edit and assemble material Duplication and distribution Examples Downloading files from on-line services Copying diskettes Stamping CD-ROMs Hardware Removable magnetic and optical disks Telecommunications Software Electronic mail Network system from "A Technical Guide to Teleconferencing and Distance Learning," 3rd edition