We believe that proposed solution will allow completely change the way in which users are consuming television content, both on-line and recorded.
Television which we know is slowly dying. It is caused by development of technology and possibility to watch media on mobile devices like tablet or smartphone and stationary like PC or MAC. Consumers are less and less interested in watching TV on-line. They prefer to record TV content and watch it later, in convenient moment or use VoD service when they want. It means that together with reducing number of TV viewers interested in classic TV will reduce income of TV operators from advertising attached to TV content.
The method to stop that trend can be Gigapixel Television, proposed by us. In opposition to 8K television (Super Hi-Vision) proposed by Japan TV operator NHK, which require to buy 8K TV Set (which is very expensive at the moment), Gigapixel TV can be watched on any TV Set, independent of its native resolution. User can use HD, 4K or 8K as well as nonstandard resolutions. It can be especially interesting in case of mobile devices which have many different resolutions.
Typical Gigapixel TV transmission should have resolution 16K (133MPixels – 15360×8640) or above. 16K picture have 64 times more pixels than Full HD picture (1920×1080). Viewer can watch Gigapixel TV scaled down to the resolution of his screen or can select fragment interesting for him and scale it up. In case of 16K transmission and HD screen scaling fragment of frame 8 times up we will obtain 1pixel of transmission equal to 1 pixel of the screen. If we will use smaller scaling few pixels of transmission will be averaged to single pixel of the screen.
Image shows proportions between Full HD, 4K, 8K and 16K. Letters are proportional to the resolution (on 4K are 2x bigger than on Full HD, on 8K are 2x bigger than on 4k and on 16K are 2x bigger than on 8K)
Imagine that wide angle, fixed Gigapixel TV camera is placed in such a way that is showing whole stadium, guest goal on the left, host goal on the right, grass, players and referees. In classic TV viewers have to see what head of transmission and cameramen will select to them. In case of Gigapixel TV viewer is making his own decision what he want to see at the moment and in what scale. He can select point on the frame (focus point) and zoom factor as he wishes to see selected part of the stadium. He can change focus point anytime or assign focus point with a player or referee or the ball. In such case scaled window, being part of the full 16K frame will follow selected object. He can switch between ball, players and/or referees easily as well as he can change the zoom factor.
What is important viewer can stop the player, scroll recorded content back and see what happened at the same time in different part of stadium. Nothing is lost because whole transmission (full frames) is always recorded. Viewer can check what did every player and referee, one after one.
Picture is showing example view from 16K camera (133Mpixels) with resolution 22000×6000 pixels. Color frames shows what will see 4 viewers on Full HD TV Sets if they will select 8x zoom (1pixel of camera = 1pixel of the screen). If the camera will have 32K resolution (530MPixels) then maximum zoom factor will be 16x and color frames on this picture will be 2x smaller.
There is also possibility of dividing TV Screen on for example 4 parts. Each part will have own focus point on the transmission frame. For example, top left window can be focused on the ball, top right window will be focused on the main referee, bottom left window will be focused on guest goalkeeper and bottom right window will be focused on center forward of the host team. That simple way we will have one sport transmission but presented by four virtual cameras. The most important is that viewer is decision maker. He decide what to see.
Exactly the same idea can be used to transmit theatrical performance or music concert. Viewer can zoom in that part of the frame he wants and focus on main character of the spectacle or on the actor playing secondary role. Also movies can be watched that way.
The biggest advantage of Gigapixel TV is total freedom of the viewer in selecting focus point and zoom factor. It follows today trend of watching what user want and when he wants. One Gigapixel TV transmission can be watched by millions of people and each of them can watch it a little differently.
Of course such solution is causing small problems. For example, advertisings, information bars or station logo cannot be sent in classic form. Those graphical elements should be sent in vector form and placed above the video by the decoder, scaled to the resolution of the user’s screen. Such graphical elements should be sent as auxiliary packets in transport stream.
Around the Gigapixel TV idea can be build new companies and services. For example, somebody have to design new remote control which will allow on intuitive control of the focus point and zoom factor by the user. There are many possible solutions. One will have touch screen with full frame displayed on it, another one will have dedicated keys for sport transmissions (different for different sports?). Automatic detection of moving objects in the frame can be implemented as well as identification of objects (e.g. players) and assignment of objects (players) to fast select keys on remote control keyboard.
For lazy viewers’ TV operator can transmit scripts prepared in real time and passed to remote control devices with ready assignment of objects (their position in the frame) to keys.
Scripts can be different for guests and for hosts, because viewers of one team may prefer to see something slightly different than viewers of other team. Viewer select script which he wants. In such case viewer do not have to control all the time focus points.
Scripts can be offered together with verbal comment which can be used as audio track for video transmission. There can be offered multiple verbal comments, in multiple languages. Viewer will select one of them, most interesting for him. In such a way viewer will see what commentator see (focus point of viewer will follow focus point of commentator). One Gigapixel TV transmission, multiple languages, multiple commentators, experts Etc.
Scripts and comments can be send together with video but can be also send independently by Internet. Thanks that enthusiasts in Internet can create own scripts and offer them to viewers of the event.
By the script we understand stream of information, synchronized with Gigapixel TV transmission by using time stamps, containing position of focus points (FP) from which user can select one or more and follow them. As a result, on the screen of TV Set will be seen part of Gigapixel TV frames, focused around the focus point. Script can also contain information about temporal zoom factor. It can be maximum zoom (for example 8x for 16K content) or lower if it will be better to show a little more. Script can also contain audio tracks in multiple languages with verbal comments. Comments can also have form of text, which will be displayed on the user’s screen or read by voice synthesizer.
Focus points can be assigned to persons and other objects (e.g. vehicles if it is a race shown) visible inside Gigapixel TV frames. Viewer can display the list of available focus points and chose one or more (if screen will be split between several windows).
Decoder can be equipped in functionality of automatic detection and identification of moving objects in the frame. In such case script is created by the decoder in real time.
Because picture analysis made by human or automatically takes time, decoder have to have ability to delay a little displayed content to allow prepare script. In case of script received with video or from Internet it should contain information about delay.
To create Gigapixel Television, new, more effective video compression is necessary. Stream have to be small enough to send it through satellite, cable TV channel or terrestrial channel.
At the moment exist only one video compression technology allowing create Gigapixel TV – HD3 developed by Patents Factory Ltd. company. Estimations shows that bitrate for 16K content should be between 9 and 17 Mbps. It is amount of data smaller than needed for 4K content coded with H264 codec.
Table shows number of pixels per frame and bitrate estimated for few example resolution and two compression technologies: H264 and HD3.