Kinect — BrandWiki

Kinect

Overview

Kinect

developer	Microsoft
type	Motion controller
generation	Seventh and eighth
release date	Xbox 360 NA: November 4, 2010 EU: November 10, 2010 COL: November 14, 2010‡R1R‡ AUS: November 18, 2010 Windows WW: February 1, 2012‡R3R‡ Xbox One WW: November 22, 2013
lifespan	2010–2017
discontinued	Windows WW: April 2, 2015‡R4R‡ Xbox 360 WW: April 20, 2016‡R5R‡ Xbox One WW: October 25, 2017
units sold

connectivity	USB 2.0 (type-A for original model; proprietary for Xbox 360 S)

platform	Xbox 360 Xbox One Windows (Windows 7 onwards)

camera	640×480 pixels @ 30 Hz (RGB camera) 640×480 pixels @ 30 Hz (IR depth-finding camera) ‡R7R‡

predecessor	Xbox Live Vision

successor	Azure Kinect

top game	Kinect Adventures

Kinect is a discontinued line of motion sensing input devices produced by Microsoft and first released in 2010. The devices generally contain RGB cameras, and infrared projectors and detectors that map depth through either structured light or time of flight calculations, which can in turn be used to perform real-time gesture recognition and body skeletal detection, among other capabilities. They also contain microphones that can be used for speech recognition and voice control.

Kinect was originally developed as a motion controller peripheral for Xbox video game consoles, distinguished from competitors (such as Nintendo's Wii Remote and Sony's PlayStation Move) by not requiring physical controllers. The first-generation Kinect was based on technology from Israeli company PrimeSense, and unveiled at E3 2009 as a peripheral for Xbox 360 codenamed "Project Natal". It was first released on November 4, 2010, and would go on to sell eight million units in its first 60 days of availability. The majority of the games developed for Kinect were casual, family-oriented titles, which helped to attract new audiences to Xbox 360, but did not result in wide adoption by the console's existing, overall userbase.

As part of the 2013 unveiling of Xbox 360's successor, Xbox One, Microsoft unveiled a second-generation version of Kinect with improved tracking capabilities. Microsoft also announced that Kinect would be a required component of the console, and that it would not function unless the peripheral is connected. The requirement proved controversial among users and critics due to privacy concerns, prompting Microsoft to backtrack on the decision. However, Microsoft still bundled the new Kinect with Xbox One consoles upon their launch in November 2013. A market for Kinect-based games still did not emerge after the Xbox One's launch; Microsoft would later offer Xbox One hardware bundles without Kinect included, and later revisions of the console removed the dedicated ports used to connect it (requiring a powered USB adapter instead). Microsoft ended production of Kinect for Xbox One in October 2017.

Kinect has also been used as part of non-game applications in academic and commercial environments, as it was cheaper and more robust than other depth-sensing technologies at the time. While Microsoft initially objected to such applications, it later released software development kits (SDKs) for the development of Microsoft Windows applications that use Kinect. In 2020, Microsoft released Azure Kinect as a continuation of the technology integrated with the Microsoft Azure cloud computing platform. Part of the Kinect technology was also used within Microsoft's HoloLens project. Microsoft discontinued the Azure Kinect developer kits in October 2023.[8][9]

History

Development

The origins of the Kinect started around 2005, at a point where technology vendors were starting to develop depth-sensing cameras. Microsoft had been interested in a 3D camera for the Xbox line earlier but because the technology had not been refined, had placed it in the "Boneyard", a collection of possible technology they could not immediately work on.[10]

In 2005, Israeli company PrimeSense was founded by mathematicians and engineers to develop the "next big thing" for video games, incorporating cameras that were capable of mapping a human body in front of them and sensing hand motions. They showed off their system at the 2006 Game Developers Conference, where Microsoft's Alex Kipman, the general manager of hardware incubation, saw the potential in PrimeSense's technology for the Xbox system. Microsoft began discussions with PrimeSense about what would need to be done to make their product more consumer-friendly: not only improvements in the capabilities of depth-sensing cameras, but a reduction in size and cost, and a means to manufacture the units at scale was required. PrimeSense spent the next few years working at these improvements.[10]

Technology

Fundamentals

The depth and motion sensing technology at the core of the Kinect is enabled through its depth-sensing. The original Kinect for Xbox 360 used structured light for this: the unit used a near-infrared pattern projected across the space in front of the Kinect, while an infrared sensor captured the reflected light pattern. The light pattern is deformed by the relative depth of the objects in front it, and mathematics can be used to estimate that depth based on several factors related to the hardware layout of the Kinect. While other structure light depth-sensing technologies used multiple light patterns, Kinect used as few as one as to achieve a high rate of 30 frames per second of depth sensing. Kinect for Xbox One switched over to using time of flight measurements. The infrared projector on the Kinect sends out modulated infrared light which is then captured by the sensor. Infrared light reflecting off closer objects will have a shorter time of flight than those more distant, so the infrared sensor captures how much the modulation pattern had been deformed from the time of flight, pixel-by-pixel. Time of flight measurements of depth can be more accurate and calculated in a shorter amount of time, allowing for more frames-per-second to be detected.[93]

Once Kinect has a pixel-by-pixel depth image, Kinect uses a type of edge detection here to delineate closer objects from the background of the shot, incorporating input from the regular visible light camera. The unit then attempts to track any moving objects from this, with the assumption that only people will be moving around in the image, and isolates the human shapes from the image.

Software

Kinect-enabled features on the Xbox operating system

Requiring at least 190 MB of available storage space,[143] Kinect system software allows users to operate Xbox 360 Dashboard console user interface through voice commands and hand gestures. Techniques such as voice recognition and facial recognition are employed to automatically identify users. Among the applications for Kinect is Video Kinect, which enables voice chat or video chat with other Xbox 360 users or users of Windows Live Messenger. The application can use Kinect's tracking functionality and Kinect sensor's motorized pivot to keep users in frame even as they move around. Other applications with Kinect support include ESPN, Zune Marketplace,[143] Netflix, Hulu Plus[144] and Last.fm.[145]

Reception

Kinect for Xbox 360

Upon its release, the Kinect garnered generally positive opinions from reviewers and critics. IGN gave the device 7.5 out of 10, saying that "Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing", though adding that for "$149.99, a motion-tracking camera add-on for Xbox 360 is a tough sell, especially considering that the entry level variation of Xbox 360 itself is only $199.99".[182] Game Informer rated Kinect 8 out of 10, praising the technology but noting that the experience takes a while to get used to and that the spatial requirement may pose a barrier.[183] Computer and Video Games called the device a technological gem and applauded the gesture and voice controls, while criticizing the launch lineup and Kinect Hub.[184]

CNET's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements.[185]

Sales

While announcing Kinect's discontinuation in an interview with Fast Co. Design on October 25, 2017, Microsoft stated that 35 million units had been sold since its release.[6] 24 million units of Kinect had been shipped by February 2013.[197] Having sold 8 million units in its first 60 days on the market, Kinect claimed the Guinness World Record of being the "fastest selling consumer electronics device".[198][199][200][201] According to Wedbush analyst Michael Pachter, Kinect bundles accounted for about half of all Xbox 360 console sales in December 2010 and for more than two-thirds in February 2011.[202]

Other motion controllers

Kinect competed with several motion controllers on other home consoles, such as Wii Remote, Wii Remote Plus and Wii Balance Board for the Wii and Wii U, PlayStation Move and PlayStation Eye for the PlayStation 3, and PlayStation Camera for the PlayStation 4.

While the Xbox 360 Kinect's controller-less nature enabled it to offer a motion-controlled experience different from the wand-based controls of the Wii and PlayStation Move, this has occasionally hindered developers from developing certain motion-controlled games that could target all three seventh-generation consoles and still provide the same experience regardless of console. Examples of seventh-generation motion-controlled games that were released on Wii and PlayStation 3, but had a version for Xbox 360 cancelled or ruled out from the start, due to issues with translating wand controls to the camera-based movement of the Kinect, include Dead Space: Extraction,[206] The Lord of the Rings: Aragorn's Quest[207]

Awards

The machine learning work on human motion capture within Kinect won the 2011 MacRobert Award for engineering innovation.[209]
Kinect Won T3's "Gadget of the Year" award for 2011.[210] It also won the "Gaming Gadget of the Year" prize.[211]
'Microsoft Kinect for Windows Software Development Kit' was ranked second in "The 10 Most Innovative Tech Products of 2011" at Popular Mechanics Breakthrough Awards ceremony in New York City.[212][213]

External links

for Xbox + Kinect
for Kinect for Windows

References

Ya llegaron a Colombia los videojuegos que no necesitan controles Caracol TV, November 14, 2010, retrieved November 14, 2010^
Microsoft to launch Kinect in Japan in November The Economic Times, September 8, 2010, retrieved October 20, 2010^
Starting February 1, 2012: Use the Power of Kinect for Windows to Change the World - Kinect for Windows Blog - Site Home - MSDN Blogs

Introduction

Kinect was first announced to the public as "Project Natal" on June 1, 2009, during Microsoft's press conference at E3 2009; film director Steven Spielberg joined Microsoft's Don Mattrick to introduce the technology and its potential.[10][13] Three demos were presented during the conference—Microsoft's Ricochet and Paint Party, and Lionhead Studios' Milo & Kate created by Peter Molyneux—while a Project Natal-enabled version of Criterion Games' Burnout Paradise was shown during the E3 exhibition.[14][15] By E3 2009, the skeletal mapping technology was capable of simultaneously tracking four people,[16][17] with a feature extraction of 48 skeletal points on a human body at 30 Hz.[17][18] Microsoft had not committed to a release date for Project Natal at E3 2009, but affirmed it would be after 2009, and likely in 2010 to stay competitive with the Wii and the PlayStation Move (Sony Interactive Entertainment's own motion-sensing system using hand-held devices).

In the months following E3 2009, rumors that a new Xbox 360 console associated with Project Natal emerged, either a retail configuration that incorporated the peripheral,[19][20] or as a hardware revision or upgrade to support the peripheral.[21][22] Microsoft dismissed the reports in public and repeatedly emphasized that Project Natal would be fully compatible with all Xbox 360 consoles. Microsoft indicated that the company considered Project Natal to be a significant initiative, as fundamental to Xbox brand as Xbox Live,[23] and with a planned launch akin to that of a new Xbox console platform.[24] Microsoft's vice president Shane Kim said the company did not expect Project Natal would extend the anticipated lifetime of the Xbox 360, which had been planned to last ten years through 2015, nor delay the launch of the successor to the Xbox 360.[25]

Following the E3 2009 show and through 2010, the Project Natal team members experimentally adapted numerous games to Kinect-based control schemes to help evaluate usability. Among these games were Beautiful Katamari and Space Invaders Extreme, which were demonstrated at Tokyo Game Show in September 2009.[26] According to Tsunoda, adding Project Natal-based control to pre-existing games involved significant code alterations, and made it unlikely that existing games could be patched through software updates to support the unit.[27] Microsoft also expanded its draw to third-party developers to encourage them to develop Project Natal games. Companies like Harmonix and Double Fine quickly took to Project Natal and saw the potential in it, and committed to developing games for the unit, such as the launch title Dance Central from Harmonix.[10]

Although its sensor unit was originally planned to contain a microprocessor that would perform operations such as the system's skeletal mapping, Microsoft reported in January 2010 that the sensor would no longer feature a dedicated processor. Instead, processing would be handled by one of the processor cores of Xbox 360's Xenon CPU.[28] Around this time, Kipmen estimated that the Kinect would only take about 10 to 15% of the Xbox 360's processing power.[29] While this was a small fraction of the Xbox 360's capabilities, industry observers believed this further pointed to difficulties in adapting pre-existing games to use Kinect, as the motion-tracking would add to a game's high computational load and exceed the Xbox 360's capabilities. These observers believed that instead the industry would develop games specific to the Kinect features.[28]

Kinect for Xbox One and decline

The Kinect release for the Xbox 360 was estimated to have sold eight million units in the first sixty days of release, earning the hardware the Guinness World Record for the "Fastest-Selling Consumer Electronics Device".[10] Over 10 million had been sold by March 2011.[10] While seemingly successful, its launch titles were primarily family-oriented games (which could be designed around Kinect's functionality and limitations), which may have drawn new audiences, but did not have the selling power of major franchises like Battlefield and Call of Duty—which were primarily designed around the Xbox 360 controller. Only an estimated 20% of the 55 million Xbox 360 owners had purchased the Kinect.[10] The Kinect team recognized some of the downsides with more traditional games and Kinect, and continued ongoing development of the unit to be released as a second-generation unit, such as reducing the latency of motion detection and improving speech recognition. Microsoft provided news of these changes to the third-party developers to help them anticipate how the improvements can be integrated into the games.[10]

Concurrent with the Kinect improvements, Microsoft's Xbox hardware team had started planning for the Xbox One around mid-2011. Part of early Xbox One specifications was that the new Kinect hardware would be automatically included with the console, so that developers would know that Kinect hardware would be available for any Xbox One, and hoping to encourage developers to take advantage of that.[10] The Xbox One was first formally announced on May 23, 2013, and shown in more detail at E3 2013 in June. Microsoft stated at these events that the Xbox One would include the updated Kinect hardware and it would be required to be plugged in at all times for the Xbox One to function. This raised concerns across the video game media: privacy advocates argued that Kinect sensor data could be used for targeted advertising, and to perform unauthorized surveillance on users. In response to these claims, Microsoft reiterated that Kinect voice recognition and motion tracking can be disabled by users, that Kinect data cannot be used for advertising per its privacy policy, and that the console would not redistribute user-generated content without permission.[47][48][49][50]

Microsoft announced in August 2013 that they had made several changes to the planned Xbox One release in response to the backlash. Among these was that the system would no longer require a Kinect unit to be plugged in to work, though it was still planned to package the Kinect with all Xbox One systems. However, this also required Microsoft to establish a US$500 price-point for the Xbox One/Kinect system at its November 2013 launch, US$100 more than the competing PlayStation 4 launched in the same time frame, which did not include any motion-sensing hardware.[10] In the months after the Xbox One release, Microsoft decided to launch a Kinect-less Xbox One system in March 2014 at the same price as the PlayStation 4, after considering that the Kinect for Xbox One had not gotten the developer support, and sales of the Xbox One were lagging due to the higher price tag of the Kinect-bundled system. Richard Irving, a program group manager that oversaw Kinect, said that Microsoft had felt that it was more important to give developers and consumers the option of developing for or purchasing the Kinect rather than forcing the unit on them.[10]

The removal of Kinect from the Xbox One retail package was the start of the rapid decline and phase-out of the unit within Microsoft. Developers like Harmonix that had been originally targeting games to use the Xbox One had put these games on hold until they knew there was enough of a Kinect install base to justify release, which resulted in a lack of games for the Kinect and reducing any consumer drive to buy the separate unit.[10] Microsoft became bearish on the Kinect, making no mention of the unit at E3 2015 and announcing at E3 2016 that the upcoming Xbox One hardware revision, the Xbox One S, would not have a dedicated Kinect port; Microsoft offered a USB adapter for the Kinect, provided free during an initial promotional period after the console's launch.[53] The more powerful Xbox One X also lacked the Kinect port and required this adapter. Even though developers still released Kinect-enabled games for the Xbox One, Microsoft's lack of statements related to the Kinect during this period led to claims that the Kinect was a dead project at Microsoft.[54][55]

Microsoft formally announced it would stop manufacturing Kinect for Xbox One on October 25, 2017.[6] Microsoft eventually discontinued the adapter in January 2018, stating that they were shifting to manufacture other accessories for the Xbox One and personal computers that were more in demand. This is considered by the media to be the point where Microsoft ceased work on the Kinect for the Xbox platform.[10][56]

Non-gaming applications and Kinect for Windows

While the Kinect unit for the Xbox platform had petered out, the Kinect was being used in academia and other applications since around 2011. The functionality of the unit along with its low US$150 cost was seen to be an inexpensive means to add depth-sensing to existing applications, offsetting the high cost and unreliability of other 3D camera options at the time. In robotics, Kinect's depth-sensing would enable robots to determine the shape and approximate distances to obstacles and maneuver around them.[57] Within the medical field, the Kinect could be used to monitor the shape and posture of a body in a quantifiable manner to enable improved health-care decisions.[58]

Around November 2010, after the Kinect's launch, scientists, engineers, and hobbyists had been able to hack into the Kinect to determine what hardware and internal software it had used, leading to users finding how to connect and operate the Kinect with Microsoft Windows and OS X over USB, which has unsecured data from the various camera elements that could be read. This further led to prototype demos of other possible applications, such as a gesture-based user interface for the operating system similar to that shown in the film Minority Report, as well as pornographic applications.[59] This mirrored similar work to hack the Wii Remote a few years earlier to use its low-cost hardware for more advanced applications beyond gameplay.[60]

Adafruit Industries, having envisioned some of the possible applications of the Kinect outside of gaming, issued a security challenge related to the Kinect, offering prize money for the successful development of an open source software development kit (SDK) and hardware drivers for the Kinect, which came to be known as Open Kinect.[61] Adafruit named the winner, Héctor Martín, by November 10, 2010,[62][63] who had produced a Linux driver that allows the use of both the RGB camera and depth sensitivity functions of the device.[64][65] It was later discovered that Johnny Lee, a core member of Microsoft's Kinect development team, had secretly approached Adafruit with the idea of a driver development contest and had personally financed it.[66] Lee had said of the efforts to open the Kinect that "This is showing us the future...This is happening today, and this is happening tomorrow."

Microsoft initially took issue with users hacking into the Kinect, stating they would incorporate additional safeguards into future iterations of the unit to prevent such hacks.[67] However, by the end of November 2010, Microsoft had turned on their original position and embraced the external efforts to develop the SDK.[68] Kipman, in an interview with NPR, said The first thing to talk about is, Kinect was not actually hacked. Hacking would mean that someone got to our algorithms that sit inside of the Xbox and was able to actually use them, which hasn't happened. Or, it means that you put a device between the sensor and the Xbox for means of cheating, which also has not happened. That's what we call hacking, and that's what we have put a ton of work and effort to make sure doesn't actually occur. What has happened is someone wrote an open-source driver for PCs that essentially opens the USB connection, which we didn't protect, by design, and reads the inputs from the sensor. The sensor, again, as I talked earlier, has eyes and ears, and that's a whole bunch of noise that someone needs to take and turn into signal.

PrimeSense along with robotics firm Willow Garage and game developer Side-Kick launched OpenNI, a not-for-profit group to develop portable drivers for the Kinect and other natural interface (NI) devices, in November 2010. Its first set of drivers named NITE were released in December 2010.[70][71] PrimeSense had also worked with Asus to develop a motion sensing device that competes with the Kinect for personal computers. The resulting product, the Wavi Xtion, was released in China in October 2011.[72][73]

Microsoft announced in February 2011 that it was planning on releasing its own SDK for the Kinect within a few months, and which was officially released on June 16, 2011, but which was limited to non-commercial uses.[74][75] The SDK enabled users to access the skeletal motion recognition system for up to two persons and the Kinect microphone array, features that had not been part of the prior Open Kinect SDK.[76] Commercial interest in Kinect was still strong, with David Dennis, a product manager at Microsoft, stating "There are hundreds of organizations we are working with to help them determine what's possible with the tech".[77] Microsoft launched its Kinect for Windows program on October 31, 2011, releasing a new SDK to a small number of companies, including Toyota, Houghton Mifflin, and Razorfish, to explore what was possible.[77] At the 2012 Consumer Electronics Show in January, Microsoft announced that it would release a dedicated Kinect for Windows unit along with the commercial SDK on February 1, 2012. The device included some hardware improvements, including support for "near mode" to recognize objects about 50 cm in front of the cameras.

With the original announcement of the revised Kinect for Xbox One in 2013, Microsoft also confirmed it would have a second generation of Kinect for Windows based on the updated Kinect technology by 2014.[80] The new Kinect 2 for Windows was launched on July 15, 2014, at a US$200 price.[81] Microsoft opted to discontinue the original Kinect for Windows by the end of 2014.[82] However, in April 2015, Microsoft announced they were also discontinuing the Kinect 2 for Windows, and instead directing commercial users to use the Kinect for Xbox One, which Microsoft said "perform identically". Microsoft stated that the demand for the Kinect 2 for Windows demand was high and difficult to keep up while also fulfilling the Kinect for Xbox One orders, and that they had found commercial developers successfully using the Kinect for Xbox One in their applications without issue.[83]

With Microsoft's waning focus on Kinect, PrimeSense was bought by Apple, Inc. in 2013, which incorporated parts of the technology into its Face ID system for iOS devices.[84][85]

Though Kinect had been cancelled, the ideas of it helped to spur Microsoft into looking more into accessibility for Xbox and its games. According to Phil Spencer, the head of Xbox at Microsoft, they received positive comments from parents of disabled and impaired children who were happy that Kinect allowed their children to play video games. These efforts led to the development of the Xbox Adaptive Controller, released in 2018, as one of Microsoft's efforts in this area.[86]

Kinect for Xbox 360 (2010)

Kinect for Xbox 360 was a combination of Microsoft built software and hardware. The hardware included a range chipset technology by Israeli developer PrimeSense, which developed a system consisting of an infrared projector and camera and a special microchip that generates a grid from which the location of a nearby object in 3 dimensions can be ascertained.[95][96][97] This 3D scanner system called Light Coding[98] employs a variant of image-based 3D reconstruction.[99][100]

The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB camera, depth sensor and microphone array running proprietary software",[101] which provide full-body 3D motion capture, facial recognition and voice recognition capabilities. At launch, voice recognition was only made available in Japan, United Kingdom, Canada and United States. Mainland Europe received the feature later in spring 2011.[102] Currently voice recognition is supported in Australia, Canada, France, Germany, Ireland, Italy, Japan, Mexico, New Zealand, United Kingdom and United States. The Kinect sensor's microphone array enables Xbox 360 to conduct acoustic source localization and ambient noise suppression, allowing for things such as headset-free party chat over Xbox Live.[103]

The depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions.[103][16] The sensing range of the depth sensor is adjustable, and Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.[18]

Described by Microsoft personnel as the primary innovation of Kinect,[104][105][106] the software technology enables advanced gesture recognition, facial recognition and voice recognition.[107] According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis with a feature extraction of 20 joints per player. However, PrimeSense has stated that the number of people the device can "see" (but not process as players) is only limited by how many will fit in the field-of-view of the camera.[108]

Reverse engineering[109] has determined that the Kinect's various sensors output video at a frame rate of ≈9 Hz to 30 Hz depending on resolution. The default RGB video stream uses 8-bit VGA resolution (640 × 480 pixels) with a Bayer color filter, but the hardware is capable of resolutions up to 1280x1024 (at a lower frame rate) and other colour formats such as UYVY. The monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels) with 11-bit depth, which provides 2,048 levels of sensitivity. The Kinect can also stream the view from its IR camera directly (i.e.: before it has been converted into a depth map) as 640x480 video, or 1280x1024 at a lower frame rate. The Kinect sensor has a practical ranging limit of 1.2 - 3.5 m distance when used with the Xbox software. The area required to play Kinect is roughly 6 m2, although the sensor can maintain tracking through an extended range of approximately 0.7 - 6 m. The sensor has an angular field of view of 57° horizontally and 43° vertically, while the motorized pivot is capable of tilting the sensor up to 27° either up or down. The horizontal field of the Kinect sensor at the minimum viewing distance of ≈0.8 m is therefore ≈87 cm, and the vertical field is ≈63 cm, resulting in a resolution of just over 1.3 mm per pixel. The microphone array features four microphone capsules[110] and operates with each channel processing 16-bit audio at a sampling rate of 16 kHz.[111]

Because the Kinect sensor's motorized tilt mechanism requires more power than the Xbox 360's USB ports can supply,[112] the device makes use of a proprietary connector combining USB communication with additional power. Redesigned Xbox 360 S models include a special AUX port for accommodating the connector,[113] while older models require a special power supply cable (included with the sensor)[110] that splits the connection into separate USB and power connections; power is supplied from the mains by way of an AC adapter.[112]

Kinect for Windows (2012)

Kinect for Windows is a modified version of the Xbox 360 unit which was first released on February 1, 2012, alongside the SDK for commercial use.[78][114] The hardware included better components to eliminate noise along the USB and other cabling paths, and improvements in the depth-sensing camera system for detection of objects at close range, as close as 50 cm, in the new "Near Mode".[78]

The SDK included Windows 7 compatible PC drivers for Kinect device. It provided Kinect capabilities to developers to build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010 and included the following features:

In March 2012, Craig Eisler, the general manager of Kinect for Windows, said that almost 350 companies are working with Microsoft on custom Kinect applications for Microsoft Windows.[116]

In March 2012, Microsoft announced that next version of Kinect for Windows SDK would be available in May 2012. Kinect for Windows 1.5 was released on May 21, 2012. It adds new features, support for many new languages and debut in 19 more countries.[117][118]

Kinect for Windows SDK for the first-generation sensor was updated a few more times, with version 1.6 released October 8, 2012,[122] version 1.7 released March 18, 2013,[123] and version 1.8 released September 17, 2013.[124]

1) Raw sensor streams: Access to low-level streams from the depth sensor, color camera sensor, and four-element microphone array.
2) Skeletal tracking: The capability to track the skeleton image of one or two people moving within Kinect's field of view for gesture-driven applications.
3) Advanced audio capabilities: Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with Windows speech recognition API.
4) Sample code and Documentation.[115]

1) Kinect for Windows 1.5 SDK would include 'Kinect Studio' a new app that allows developers to record, playback, and debug clips of users interacting with applications.[119]
2) Support for new "seated" or "10-joint" skeletal system that will let apps track the head, neck, and arms of a Kinect user—whether they're sitting down or standing; which would work in default and near mode.
3) Support for four new languages for speech recognition – French, Spanish, Italian, and Japanese. Additionally it would add support for regional dialects of these languages along with English.[120]
4) It would be available in Hong Kong, South Korea, and Taiwan in May and Austria, Belgium, Brazil, Denmark, Finland, India, the Netherlands, Norway, Portugal, Russia, Saudi Arabia, Singapore, South Africa, Sweden, Switzerland and the United Arab Emirates in June.[121]

Kinect for Xbox One (2013)

An upgraded iteration of Kinect was released on November 22, 2013, for Xbox One. It uses a wide-angle time-of-flight camera, and processes 2 gigabits of data per second to read its environment. The new Kinect has greater accuracy with three times the fidelity over its predecessor and can track without visible light by using an active IR sensor. It has a 60% wider field of vision with a minimum working distance of 0.91 m away from the sensor, compared to 1.83 m for the original Kinect,[125] and can track up to 6 skeletons at once. It can also detect a player's heart rate, facial expression, the position and orientation of 25 individual joints (including thumbs), the weight put on each limb, speed of player movements, and track gestures performed with a standard controller. The color camera captures 1080p video that can be displayed in the same resolution as the viewing screen, allowing for a broad range of scenarios. In addition to improving video communications and video analytics applications, this provides a stable input on which to build interactive applications. Kinect's microphone is used to provide voice commands for actions such as navigation, starting games, and waking the console from sleep mode.[126][127] The recommended player's height is at least 40 inches, which roughly corresponds to children of 4 1/2 years old and up.[128][129]

All Xbox One consoles were initially shipped with Kinect included.[52] In June 2014, bundles without Kinect were made available,[130] along with an updated Xbox One SDK allowing game developers to explicitly disable Kinect skeletal tracking, freeing up system resources that were previously reserved for Kinect even if it was disabled or unplugged.[130][131] As interest in Kinect waned in 2014, later revisions of the Xbox One hardware, including the Xbox One S and Xbox One X, dropped the dedicated Kinect port, requiring users to purchase a USB 3.0 and AC adapter to use the Kinect for Xbox One.[132][133]

A standalone Kinect for Xbox One, bundled with a digital copy of Dance Central Spotlight, was released on October 7, 2014.[134]

Considered a market failure compared to the Kinect for Xbox 360, the Kinect for Xbox One product was discontinued by October 25, 2017. Production of the adapter cord also ended by January 2018.[135]

Non-video game uses of Kinect

Numerous developers are researching possible applications of Kinect that go beyond the system's intended purpose of playing games, further enabled by the release of the Kinect SDK by Microsoft.[159]

For example, Philipp Robbel of MIT combined Kinect with iRobot Create to map a room in 3D and have the robot respond to human gestures,[160] while an MIT Media Lab team is working on a JavaScript extension for Google Chrome called depthJS that allows users to control the browser with hand gestures.[161] Other programmers, including Robot Locomotion Group at MIT, are using the drivers to develop a motion-controller user interface similar to the one envisioned in Minority Report.[162] The developers of MRPT have integrated open source drivers into their libraries and provided examples of live 3D rendering and basic 3D visual SLAM.[163] Another team has shown an application that allows Kinect users to play a virtual piano by tapping their fingers on an empty desk.[164] Oliver Kreylos, a researcher at University of California, Davis, adopted the technology to improve live 3-dimensional videoconferencing, which NASA has shown interest in.[165]

Alexandre Alahi from EPFL presented a video surveillance system that combines multiple Kinect devices to track groups of people even in complete darkness.[166] Companies So touch and Evoluce have developed presentation software for Kinect that can be controlled by hand gestures; among its features is a multi-touch zoom mode.[167] In December 2010, the free public beta of HTPC software KinEmote was launched; it allows navigation of Boxee and XBMC menus using a Kinect sensor.[168] Soroush Falahati wrote an application that can be used to create stereoscopic 3D images with a Kinect sensor.[169]

In human motion tracking, Kinect might suffer from occlusion which is when some human body joints are occluded and cannot be tracked accurately by Kinect's skeletal model.[170] Therefore, fusing its data with other sensors can provide a more robust tracking of the skeletal model. For instance, in a study, an Unscented Kalman filter (UKF) was used to fuse Kinect 3D position data of shoulder, elbow, and wrist joints to those obtained from two inertial measurement units (IMUs) placed on the upper and lower arm of a person.[171] The results showed an improvement of up to 50% in the accuracy of the position tracking of the joints. In addition to solving the occlusion problem, as the sampling frequency of the IMUs was 100 Hz (compared to ~30 Hz for Kinect), the improvement of skeletal position was more evident during fast and dynamic movements.

Kinect also shows compelling potential for use in medicine. Researchers at the University of Minnesota have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder.[172] Several groups have reported using Kinect for intraoperative, review of medical imaging, allowing the surgeon to access the information without contamination.[173][174] This technique is already in use at Sunnybrook Health Sciences Centre in Toronto, where doctors use it to guide imaging during cancer surgery.[175] At least one company, GestSure Technologies, is pursuing the commercialization of such a system.[176]

NASA's Jet Propulsion Laboratory (JPL) signed up for the Kinect for Windows Developer program in November 2013 to use the new Kinect to manipulate a robotic arm in combination with an Oculus Rift virtual reality headset, creating "the most immersive interface" the unit had built to date.[177]

Kinect for Xbox One

Although featuring improved performance over the original Kinect, its successor has been subject to mixed responses. In its Xbox One review, Engadget praised Xbox One's Kinect functionality, such as face recognition login and improved motion tracking, but said that while the device was "magical", "every false positive or unrecognized [voice] command had us reaching for the controller."[194] The Kinect's inability to understand some accents in English was criticized.[195] Writing for Time, Matt Peckham described the device as being "chunky" in appearance, but that the facial recognition login feature was "creepy but equally sci-fi-future cool", and that the new voice recognition system was a "powerful, addictive way to navigate the console, and save for a few exceptions that seem to be smoothing out with use". However, its accuracy was found to be affected by background noise, and Peckham further noted that launching games using voice recognition required that the full title of the game be given rather than an abbreviated name that the console "ought to semantically understand", such as Forza Motorsport 5 rather than "Forza 5".[196]

Prior to Xbox One's launch, privacy concerns were raised over the new Kinect; critics showed concerns the device could be used for surveillance, stemming from the originally announced requirements that Xbox One's Kinect be plugged in at all times, plus the initial always-on DRM system that required the console to be connected to the internet to ensure continued functionality. Privacy advocates contended that the increased amount of data which could be collected with the new Kinect (such as a person's eye movements, heart rate, and mood) could be used for targeted advertising. Reports also surfaced regarding recent Microsoft patents involving Kinect, such as a DRM system based on detecting the number of viewers in a room, and tracking viewing habits by awarding achievements for watching television programs and advertising. While Microsoft stated that its privacy policy "prohibit[s] the collection, storage, or use of Kinect data for the purpose of advertising", critics did not rule out the possibility that these policies could be changed prior to the release of the console. Concerns were also raised that the device could also record conversations, as its microphone remains active at all times. In response to the criticism, a Microsoft spokesperson stated that users are "in control of when Kinect sensing is On, Off or Paused", will be provided with key privacy information and settings during the console's initial setup, and that user-generated content such as photos and videos "will not leave your Xbox One without your explicit permission."[47][48][49][50] Microsoft ultimately decided to reverse its decision to require Kinect usage on Xbox One, but the console still shipped with the device upon its launch in November 2013.[52]

Kinect

History

Development

Technology

Fundamentals

Software

Kinect-enabled features on the Xbox operating system

Reception

Kinect for Xbox 360

Sales

Other motion controllers

Awards

See also

External links

References

Introduction

Kinect for Xbox 360 marketing and launch

Kinect for Xbox One and decline

Non-gaming applications and Kinect for Windows

Integrating Kinect with Microsoft Azure

Kinect for Xbox 360 (2010)

Kinect for Windows (2012)

Kinect for Xbox One (2013)

Kinect 2 for Windows (2014)

Azure Kinect (2019)

Video games

KinectShare.com

Kinect Fun Labs

Non-video game uses of Kinect

Kinect for Xbox One