Photosynth is a discontinued app and service from

Microsoft Live Labs Microsoft Live Labs was a partnership between MSN and Microsoft Research that focused on applied research for Internet products and services at Microsoft. Live Labs was headed by Dr. Gary William Flake, who prior to joining Microsoft was a princip ...

and the

University of Washington The University of Washington (UW and informally U-Dub or U Dub) is a public research university in Seattle, Washington, United States. Founded in 1861, the University of Washington is one of the oldest universities on the West Coast of the Uni ...

that analyzes

digital photograph Digital photography uses cameras containing arrays of electronics, electronic photodetectors interfaced to an analog-to-digital converter (ADC) to produce images focused by a lens (optics), lens, as opposed to an exposure on photographic film. ...

s and generates a three-dimensional model of the photos and a

point cloud A point cloud is a discrete set of data Point (geometry), points in space. The points may represent a 3D shape or object. Each point Position (geometry), position has its set of Cartesian coordinates (X, Y, Z). Points may contain data other than ...

of a photographed object. Pattern recognition components compare portions of images to create points, which are then compared to convert the image into a model. Users are able to view and generate their own models using a software tool available for download at the Photosynth website.

History

Photosynth is based on Photo Tourism, a research project by University of Washington graduate student Noah Snavely. Shortly after Microsoft's acquisition of Seadragon in early 2006, that team began work on Photosynth, under the direction of Seadragon founder

Blaise Agüera y Arcas Blaise Agüera y Arcas (born August 28, 1975) is an American artificial intelligence (AI) researcher, software engineer, software architect, and author. He is vice president and fellow at Google Research, where he leads a team that conducts basi ...

. Microsoft released a free tech preview version on November 9, 2006. Users could view models generated by Microsoft or the

BBC The British Broadcasting Corporation (BBC) is a British public service broadcaster headquartered at Broadcasting House in London, England. Originally established in 1922 as the British Broadcasting Company, it evolved into its current sta ...

, but not create their own models at that time. Microsoft teamed up with

NASA The National Aeronautics and Space Administration (NASA ) is an independent agencies of the United States government, independent agency of the federal government of the United States, US federal government responsible for the United States ...

on August 6, 2007 allowing users to preview its Photosynth technology showing the

Space Shuttle Endeavour Space Shuttle ''Endeavour'' (Orbiter Vehicle Designation: OV-105) is a retired Space Shuttle orbiter, orbiter from NASA's Space Shuttle program and the fifth and final operational Space Shuttle, Shuttle built. It embarked on its first mission, ...

. On August 20, 2007, a preview showing the tiles of Endeavour during the backflip process was made available for viewing. On August 20, 2008, Microsoft officially released Photosynth to the public, allowing users to upload their images and generate their own Photosynth models. In March 2010, Photosynth added support for

gigapixel A gigapixel image is a digital image bitmap composed of one billion (109) pixels (picture elements), 1000 times the information captured by a 1 megapixel digital camera. A square image of 31,623 pixels in width and height is one gigapixel. Cur ...

panorama A panorama (formed from Greek language, Greek πᾶν "all" + ὅραμα "view") is any Obtuse angle, wide-angle view or representation of a physical space, whether in painting, drawing, photography (panoramic photography), film, seismic image ...

s stitched in Microsoft ICE. The panoramas use Seadragon based technology similar to the system already used in synths. In July 2015, Microsoft announced it would be retiring the Photosynth mobile apps. As Photosynth prepared to shut down in early 2017,

Mapillary Mapillary is a service for open-source sharing of crowdsourced geotagged photos, developed by remote company Mapillary AB, based in Malmö, Sweden. Mapillary was launched in 2013 and acquired by Meta Platforms, Inc. in 2020. It offers street lev ...

, a crowdsourced street-level imaging platform, reached out to the Photosynth community with their Photosynth-to-Mapillary blog post, and the official Photosynth Twitter account suggested users "check them out". On February 6, 2017, Microsoft decommissioned the Photosynth website and services. On December 20, 2017, Photosynth returned as a feature of the

Microsoft Pix Microsoft Pix is a camera phone application developed by Microsoft Research for iOS. Microsoft Research announced Pix in July, 2016, calling it an "intelligent camera app".Killian Bell, "Microsoft says its new camera app is even better than Apple� ...

app. In the development of Microsoft Flight Simulator, Microsoft's Photosynth technology returned to recreate buildings and terrain across the entire world.

Process

The Photosynth technology works in two steps. The first step involves the analysis of multiple photographs taken of the same area. Each photograph is processed using an

interest point detection In computer vision and image processing, a feature is a piece of information about the content of an image; typically about whether a certain region of the image has certain properties. Features may be specific structures in the image such as ...

and matching algorithm developed by

Microsoft Research Microsoft Research (MSR) is the research subsidiary of Microsoft. It was created in 1991 by Richard Rashid, Bill Gates and Nathan Myhrvold with the intent to advance state-of-the-art computing and solve difficult world problems through technologi ...

which is similar in function to

UBC The University of British Columbia (UBC) is a Public university, public research university with campuses near University of British Columbia Vancouver, Vancouver and University of British Columbia Okanagan, Kelowna, in British Columbia, Canada ...

Scale-invariant feature transform The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local '' features'' in images, invented by David Lowe in 1999. Applications include object recognition, robotic mapping and navigation, ...

. This process identifies specific features, for example the corner of a window frame or a door handle. Features in one photograph are then compared to and matched with the same features in the other photographs. Thus photographs of the same areas are identified. By analyzing the position of matching features within each photograph, the program can identify which photographs belong on which side of others. By analyzing subtle differences in the relationships between the features (angle, distance, etc.), the program identifies the 3D position of each feature, as well as the position and angle at which each photograph was taken. This process is known scientifically as

bundle adjustment In photogrammetry and computer stereo vision, bundle adjustment is simultaneous refining of the 3D coordinates describing the scene geometry, the parameters of the relative motion, and the optical characteristics of the camera(s) employed to acq ...

and is commonly used in the field of

photogrammetry Photogrammetry is the science and technology of obtaining reliable information about physical objects and the environment through the process of recording, measuring and interpreting photographic images and patterns of electromagnetic radiant ima ...

, with similar products available such as Imodeller and D-Sculptor. This first step is extremely computationally intensive, but only has to be performed once on each set of photographs. The second step involves the display of and navigation through the 3D point cloud of features identified in the first step. This is done with the publicly downloadable Photosynth viewer. The viewer resides on a client computer and maintains a connection to a server that stores the original photographs. It enables a user to, among other things, see any of the photographs from their original vantage point. It incorporates

DeepZoom Deep Zoom is a technology developed by Microsoft for efficiently transmitting and viewing images. It allows users to pan around and zoom in on a large, high resolution image or a large collection of images. It reduces the time required for initial ...

technology Microsoft obtained through its acquisition of Seadragon in January 2006. The Seadragon technology enables smooth zooming into the high-resolution photographs without downloading them to the user's machine. The Photosynth

Direct 3D Direct3D is a graphics application programming interface (API) for Microsoft Windows. Part of DirectX, Direct3D is used to render three-dimensional graphics in applications where performance is important, such as games. Direct3D uses hardware a ...

-based viewing software is only available to the

Windows 7 Windows 7 is a major release of the Windows NT operating system developed by Microsoft. It was Software release life cycle#Release to manufacturing (RTM), released to manufacturing on July 22, 2009, and became generally available on October 22, ...

Windows Vista Windows Vista is a major release of the Windows NT operating system developed by Microsoft. It was the direct successor to Windows XP, released five years earlier, which was then the longest time span between successive releases of Microsoft W ...

and

Windows XP Windows XP is a major release of Microsoft's Windows NT operating system. It was released to manufacturing on August 24, 2001, and later to retail on October 25, 2001. It is a direct successor to Windows 2000 for high-end and business users a ...

operating systems. However, the team released a

Silverlight Microsoft Silverlight is a discontinued application framework designed for writing and running rich internet applications, similar to Adobe's runtime, Adobe Flash. While early versions of Silverlight focused on streaming media, later version ...

version of the viewer which has succeeded the D3D viewer as the main option to view photosynths.

Mobile apps

As of March 2009, user uploaded Photosynth collections were available for viewing on

iPhone The iPhone is a line of smartphones developed and marketed by Apple that run iOS, the company's own mobile operating system. The first-generation iPhone was announced by then–Apple CEO and co-founder Steve Jobs on January 9, 2007, at ...

s using iSynth (3D) or Seadragon Mobile (2D only). The Photosynth application was also available from the

App Store An app store, also called an app marketplace or app catalog, is a type of digital distribution platform for computer software called applications, often in a mobile context. Apps provide a specific set of functions which, by definition, do not i ...

to download on

iPod Touch The iPod Touch (stylized as iPod touch) is a discontinued line of iOS-based mobile devices designed and formerly marketed by Apple Inc. with a touchscreen-controlled user interface. As with other iPod models, the iPod Touch can be used as a po ...

and

. In May 2012, Microsoft released a Photosynth App for its mobile platform,

Windows Phone Windows Phone (WP) is a discontinued mobile operating system developed by Microsoft Mobile for smartphones as the replacement successor to Windows Mobile and Zune. Windows Phone featured a new user interface derived from the Metro design languag ...

. On July 10, 2015, Microsoft announced that they are retiring the Photosynth Mobile Apps, removing them from their stores, and are no longer supporting or updating them. While the Photosynth platform was shut down in early 2017, its features re-appeared in the fall within the Microsoft Pix app for iOS, however as of late 2020, the Photosynth features appear to no longer be part of the Microsoft Pix app.

Capabilities

The latest generation of photosynths are easy to capture, as photographs taken by any regular digital camera or mobile phone can be uploaded to Photosynth. Users have the option to

geotag Geotagging, or GeoTagging, is the process of adding geographical identification metadata to various media such as a geotagged photograph or video, websites, SMS messages, QR Codes or RgSSfeeds and is a form of geospatial metadata. This data us ...

their digital shots on sites such as

Flickr Flickr ( ) is an image hosting service, image and Online video platform, video hosting service, as well as an online community, founded in Canada and headquartered in the United States. It was created by Ludicorp in 2004 and was previously a co ...

and then upload them to the online Photosynth web service. Images uploaded on Photosynth give people the ability to seamlessly view landmarks, public spaces and objects from all sides. *Walk or fly through a scene to see photos from any angle *Zoom in or out of a photo *See where pictures were taken in relation to one another *Smoothly change viewing angle between nearby photos *Smoothly zoom in and out of high-resolution photos *Find similar photos to the one you're currently viewing

In the media

* On April 30, 2008, Photosynth was featured in the episode '' Admissions'' of the television show CSI: New York. *

CNN Cable News Network (CNN) is a multinational news organization operating, most notably, a website and a TV channel headquartered in Atlanta. Founded in 1980 by American media proprietor Ted Turner and Reese Schonfeld as a 24-hour cable ne ...

utilized Photosynth for a user-contributed 3D vision of the inauguration of Barack Obama as the President of the United States. * In the Angels & Demons "Path of Illumination Contest," Photosynth is used as well as advertised in the website. * In the

MSN MSN is a web portal and related collection of Internet services and apps provided by Microsoft. The main webpage provides news, weather, sports, finance and other content curated from hundreds of different sources that Microsoft has partnere ...

's coverage of the 70th anniversary of the RAF victory in the

Battle of Britain The Battle of Britain () was a military campaign of the Second World War, in which the Royal Air Force (RAF) and the Fleet Air Arm (FAA) of the Royal Navy defended the United Kingdom (UK) against large-scale attacks by Nazi Germany's air force ...

, a Photosynth hangar of some of the fighter planes from the battle was built. A series of Photosynths have also been created for the

Imperial War Museum Duxford Imperial War Museum Duxford, also known as IWM Duxford or simply Duxford, is a branch of the Imperial War Museum near Duxford in Cambridgeshire, England. Duxford, Britain's largest aviation museum, houses exhibits, including nearly 200 aircraf ...

so that web users could experience the planes up close from a range of angles.

References

External links

Official University of Washington Photo Tourism Site
{{Microsoft Research Labs, state=uncollapsed Microsoft Live Labs Photo software Photo stitching software 3D graphics software Panorama software Virtual reality Discontinued Microsoft software Photogrammetry software