Research Topics

- Image Inpainting
- Diminished Reality
- Generation of Omnidirectional Video without Invisible Areas
- Surface Completion
- Super Resolution

(Japanese)

Image Inpainting (Image Completion)

Unneeded objects are removed from an image by minimizing an energy function based on the similarity of textures.

Original Missing regions (Red) Result
Original Missing regions (Red) Result
Original Missing regions (Red) Result

Video of inpainting process

More results are here.
You can use input images for your experiments.

publications

Hiroto Sasao, Norihiko Kawai, Tomokazu Sato, and Naokazu Yokoya:
"A study on effect of automatic perspective correction on exemplar-based image inpainting",
ITE Trans. on Media Technology and Applications, Vol. 4, No. 1, pp. 21-32, Jan. 2016. (pdf file)

N. Kawai and N. Yokoya:
"Image inpainting considering symmetric patterns",
Proc. 21st IAPR Int. Conf. on Pattern Recognition (ICPR2012), pp. 2744-2747, Nov. 2012. (pdf file)

Norihiko Kawai, Tomokazu Sato and Naokazu Yokoya:
"Image Inpainting Considering Brightness Change and Spatial Locality of Textures and Its Evaluation",
Proc. Pacific-Rim Symposium on Image and Video Technology (PSIVT2009), Jan. 2009. (pdf file)

Norihiko Kawai, Tomokazu Sato and Naokazu Yokoya:
"Image Inpainting Considering Brightness Change and Spatial Locality of Textures",
Proc. Int. Conf. on Computer Vision Theory and Applications (VISAPP), Vol. 1, pp. 66-73, Jan. 2008. (pdf file)




Diminished Reality

Real objects in video are removed in real time.

Video of Diminished Reality

Publications

N. Kawai, T. Sato, and N. Yokoya:
"Diminished reality based on image inpainting considering background geometry",
IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 3, pp. 1236-1247, March 2016. (PDF(Author version))(IEEE Xplore)
(Also presented in SIGGGRAPH ASIA 2016, IEEE TVCG Invited Session)

N. Kawai, T. Sato, and N. Yokoya:
"Diminished Reality Considering Background Structures",
Proc. IEEE International Symposium on Mixed and Augumented Reality (ISMAR2013), pp. 259-260, Oct. 2013. (pdf file)
Award: Pest Poster - Honourable Mention of ISMAR2013

Norihiko Kawai
"Diminished Reality Based on Image Inpainting for Visually Removing Real Objects in Real Time",
Proc. International Display Workshops, pp. 1296-1299, Dec. 2014.
IDW '14 Best Demonstration Award


Diminished reality for AR marker hiding

Video for marker hiding without planar assumption
publications

N. Kawai, T. Sato, N. Yuta and N. Yokoya:
"Augmented Reality Marker Hiding with Texture Deformation",
IEEE Transactions on Visualization and Computer Graphics, Vol. 23, No. 10, pp. 2288-2300, Oct. 2017. (PDF(Author Version))(IEEE Xplore)


Video of marker hiding

Publications

N. Kawai, M. Yamasaki, T. Sato, and N. Yokoya:
"Diminished Reality for AR Marker Hiding Based on Image Inpainting with Reflection of Luminance Changes",
ITE Transactions on Media Technology and Applications, Vol. 1, No. 4, pp. 343-353, Oct. 2013. (pdf file)

N. Kawai, M. Yamasaki, T. Sato, and N. Yokoya:
"AR marker hiding based on image inpainting and reflection of illumination changes",
Proc. 11th IEEE and ACM Int. Sympo. on Mixed and Augmented Reality (ISMAR2012), pp. 293-294, Nov. 2012. (pdf file)




Video completion (Video Inpainting) for omnidirectional video

Background estimation
Publications

N. Kawai, N. Inoue, T. Sato, F. Okura, Y. Nakashima, and N. Yokoya:
"Background estimation for a single omnidirectional image sequence captured with a moving camera",
IPSJ Trans. on Computer Vision and Applications, Vol. 6, pp. 68-72, July 2014. (pdf file)



Method for only camera's blind spot
Telepresence

publications

Norihiko Kawai, Kotaro Machikita, Tomokazu Sato and Naokazu Yokoya:
"Video Completion for Generating Omnidirectional Video without Invisible Areas",
IPSJ Transactions on Computer Vision and Applications, Vol. 2, pp. 200-213, Nov. 2010. (pdf file)

Norihiko Kawai, Kotaro Machikita, Tomokazu Sato and Naokazu Yokoya:
"Generation of an Omnidirectional Video without Invisible Areas Using Image Inpainting",
Proc. Asian Conf. on Computer Vision (ACCV2009), pp. 359-370, Sep. 2009. (pdf file)



Surface Completion (3D Inpainting)

3D mesh models generated with a range scanner or video images often have holes due to occlusions.
The missing regions in 3D mesh models are filled in by minimizing an energy function based on the similarity of local shapes.

Completion of shape

3D model with a missing regionCompleted model


Video of completion process

publications

Norihiko Kawai, Tomokazu Sato and Naokazu Yokoya:
"Efficient Surface Completion Using Principal Curvature and Its Evaluation",
IEEE Int. Conf. on Image Processing (ICIP2009), pp. 521-524, Nov. 2009. (pdf file)

Norihiko Kawai, Tomokazu Sato and Naokazu Yokoya:
"Surface Completion by Minimizing Energy Based on Similarity of Shape",
Proc. IEEE Int. Conf. on Image Processing (ICIP2008), pp. 1532-1535, Oct. 2008. (pdf file)


Completion of shape and texture

3D model with a missing regionCompleted model
publications

N. Kawai, A. Zakhor, T. Sato, and N. Yokoya:
"Surface completion of shape and texture based on energy minimization",
Proc. Int. Conf. on Image Processing (ICIP), pp. 913-916, Sep. 2011. (pdf file)


Super-Resolution

Super-Resolution for Stereo Video

A zoomed stereo video is generated by optically zooming one camera and cutting out and super-resolving the other camera image.
Right-eye imageLeft-eye image
Input
Output

Y. Hayashi, N. Kawai, T. Sato, M. Okumoto, and N. Yokoya:
"Generation of a zoomed stereo video using two synchronized videos with different magnifications",
IEICE Trans. on Information and Systems, Vol. E98-D, No. 9, pp. 1691-1701, Sep. 2015. (pdf file)

Y. Hayashi, N. Kawai, T. Sato, M. Okumoto, and N. Yokoya:
"Generation of a super-resolved stereo video using two synchronized videos with different magnifications",
Proc. 6th Pacific-Rim Sympo. on Image and Video Technology (PSIVT2013), pp. 194-205, Oct. 2013. (pdf file)

Exemplar-based Super-Resolution using Novel View Synthesis

Y. Hayashi, N. Kawai, T. Sato, and N. Yokoya:
"Image resolution enhancement based on novel view synthesis",
Proc. IEEE Int. Conf. on Image Processing (ICIP2015), 5 pages, Sep. 2015. (pdf file)

Spatio-Temporal Super-Resolution Using Depth Maps

Temporal and spatial resolution is increased by using depth maps.

Video

Publications

Yusaku Awatsu, Norihiko Kawai, Tomokazu Sato and Naokazu Yokoya:
"Spatio-Temporal Super-Resolution Using Depth Map",
Proc. 16th Scandinavian Conference on Image Analysis (SCIA2009), pp. 696-705 June. 2009. (pdf file)



Back to Toppage