Keynotes

Professor Nadia Magnenat Thalmann

Professor Nadia Magnenat Thalmann is Director of MIRALab at the University of Geneva in Switzerland, a ground- breaking interdisciplinary research Institute she founded in 1989. Professor Nadia Magnenat Thalmann has pioneered research in 3D physical modeling of clothes and early facial animation models. She has initiated fundamental research in VR/AR/MR showing Mixed Reality scenes in Pompeii and in cultural applications. Her work in 3D simulation of medical articulations, visualizing the hip’s cartilage of classical ballerinas while dancing was highly recognized. In NTU, Singapore, she revolutionized social robotics by unveiling the first realistic social robot Nadine that can show mood and emotions and remember people and actions. Besides having bachelor’s and master’s degrees in several disciplines, Professor Thalmann completed her PhD in quantum physics at the University of Geneva. She has received honorary doctorates from Leibniz University of Hannover and the University of Ottawa and several other prestigious Awards. She is a life Member of the Swiss Academy of Engineering Sciences. 

Overview of Nadia's talk:

Title: Digital and Robotic humanoid twins: for which purposes?

In this talk, I will start to explain my early motivation to develop realistic virtual humans, in particular some pioneer work in creating 3D famous legendary stars as Marilyn and Humphrey (1987). I will show the development of the virtual humans as making hair, clothes, facial and body animation, and how they can be used in various simulations and settings. Particularly, I will show anatomical models of see-through virtual patients where some body parts as the hip could be modelled in 3D according to the real anatomy of the person.

I will also address the creation and use of humanoid robots. I will discuss the robot Nadine that is my twin and why I chose to have this twin. I will discuss and demonstrate how we can include a humanoid robot in different contexts as insurances, elderly homes, or Museums for education. I will conclude by enumerating further research that is necessary for having social robots or digital twins as companions in many different environments.

Florian Fuchs

Florian Fuchs is an AI engineer at Sony AI Zurich. His work focuses on applying Reinforcement Learning to interactive and dynamic games in order to enhance the gaming experience and support game developers to unleash their creativity. Florian holds an MSc in computer science with a focus on machine learning at the University of Zurich. After his master thesis, where he first achieved super-human time trial results in the highly realistic racing simulator "Gran Turismo SPORT" using end-to-end Deep Reinforcement Learning, he was then part of the Sony AI team who developed the first racing agents competitive with the world’s best e-sports drivers.

Overview of Florian's talk:

Title: Design of Superhuman Racing AI agent, Gran Turismo Sophy, trained through Deep Reinforcement Learning

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical maneuvers to pass or block opponents while operating their vehicles at their traction limits. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the nonlinear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world’s best e-sports drivers. We combine state-of-the-art model-free deep reinforcement learning algorithms with mixed scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing’s important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we illuminate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

Ali Israr

Ali Israr is an engineer, researcher, leading haptics research and development. Ali’s interests are in the field of haptics and physical sensory feedback, and explores the use of "touch feedback" technologies in entertainment, assistive, educational, social, therapeutic, virtual and everyday settings. Ali has engineered tools and technologies for interactive sensory experiences, developed solutions for seamless flow of sensory information between a user and their devices, and authored technical publications in numerous conferences and journals. In the past, Ali led haptics research in Meta Platforms Inc (formerly, Facebook Inc., USA) and as an Imagineer in Disney Research, USA. Ali received his B.Sc degree from UET, Pakistan and MS and PhD degrees in Mechanical Engineering from Purdue University, USA.

Overview of Ali's talk:

Title: Haptic Media in XR Horizons

Haptic feedback is a powerful sensory medium for information communication, yet current XR technologies underutilize the capacity and capabilities of haptic systems in everyday applications. In today’s consumer industry, simple vibrotactile actuators are commonly incorporated in handheld devices, controllers and wristbands to provide feedback for low information throughput of tactile contacts, impacts, icons, button clicks, and surface texture. Complex and dynamic haptic events, such as object stiffness, shape, and weight during real and virtual object manipulations; spatiotemporal tactile guidance for instructions and user navigation; rich tactile messages for language and interpersonal communication; and haptic feedback for user wellness and well-being, are limited to academic investigations and institutional research. In this talk, I will present background on haptic media and information systems, and recent tools and processes for haptic media management, authoring and sharing. I will also highlight current challenges and new horizons for haptic media and finally I will conclude with ethical, privacy and piracy issues related to Haptic Media in XR.

Dr. Morgan McGuire

Dr. Morgan McGuire is the Chief Scientist at Roblox, building a creative, safe, civil, and scalable Metaverse. Roblox combines social interaction with a dynamic 3D environment and economy, with research impact spanning both technology and social sciences.

Morgan contributed to scientific publications that span from compilers and networking to 3D graphics; textbooks including “the bible” of 3D, Computer Graphics: Principles & Practice 3rd Edition, The Graphics Codex, Creating Games; the NVIDIA RTXGI and Reflex SDKs, Skylanders®, Call of Duty®, Marvel Ultimate Alliance®, and Titan Quest® video games series; and the open source G3D Innovation Engine and Markdeep document system.

Morgan holds current faculty positions at the University of Waterloo and McGill University and was previously a full professor at Williams College and Director of Hyperscale Graphics Research at NVIDIA.

Overview of Morgan's talk:

Title: The Network is the Renderer

Roblox is a global platform for 3D coexperience. Over 17 years it grew to 55 million daily active users across console, desktop, mobile, and VR in 180 countries. In this talk I present some of the unique technical challenges for this system's present and future. These arise due to simultaneously working with media streaming, deep neural networks, and real-time 3D, all at global scale and applied to user-generated content. For some of these challenges I will describe the solutions created and deployed at Roblox. The others are important new areas of investigation I pose to our Research division and the multimedia systems community. Common among these is a dramatic shift in how we think about media. What were once separate synthesis and transport challenges for audio-visual data will become a multinode distributed system problem where the data have new intermediate forms and synthesis and transport are inseparable. 

Dr. Mohamed Hefeeda

Mohamed Hefeeda is a Professor in the School of Computing Science at Simon Fraser University (SFU), Canada, where has been serving as the Director of the School since May 2018. He founded and leads the Network and Multimedia Systems Lab (http://nmsl.cs.sfu.ca) at SFU. His research interests include multimedia systems, mobile and wireless video streaming, immersive video processing and delivery, and network systems and protocols. Dr. Hefeeda received one of the prestigious NSERC Discovery Accelerator Supplements (DAS) awards in 2011, which are granted to a select group of distinguished researchers from all Science and Engineering disciplines in Canada. His research on mobile multimedia systems has resulted in multiple patents and conference awards (e.g., ACM MMSys Best Paper, ACM Multimedia Best Demo, and IEEE Innovation Best Paper), and has been featured in several news venues, including ACM Tech News, World Journal News, and CTV British Columbia.

Dr. Hefeeda has served on the editorial boards of premier journals such as the ACM Transactions on Multimedia Computing, Communications and Applications (TOMM), where he was named the Best Associate Editor in 2014. He has served on the organization committees and/or co/chaired several international conferences such as ACM MMSys, ACM MM, ICME, and NOSSDAV.

Dr. Hefeeda received his Ph.D. from Purdue University, USA in 2004, and M.Sc. and B.Sc. from Mansoura University, Egypt in 1997 and 1994, respectively.

Overview of Mohamed's talk:

Title: DeepGame: Efficient Video Encoding for Cloud Gaming

Cloud gaming (CG) has seen increasing popularity in recent years, as it offers faster and wider deployment of games--users can play games on virtually any device and without having to download and install the large software packages of games. Current popular CG services include Sony PlayStation Now, Google Stadia, Microsoft Xbox Cloud Gaming, and Amazon Tempo. CG offloads the game rendering and encoding to cloud datacenters. Games are then delivered as video streams to players. Delivering high-quality games to millions of players, however, poses a major challenge for CG platforms, because of the substantial bandwidth requirements and the stringent latency constraints. In this talk, I will present a new video encoding pipeline, called DeepGame, for CG platforms to reduce the bandwidth requirements. DeepGame learns the player’s contextual interest in the game and the temporal correlation of that interest using a spatiotemporal deep neural network. Then, it encodes various areas in the video frames with different quality levels proportional to their contextual importance, while maintaining a smooth quality within each frame and across successive frames. DeepGame runs in real time and achieves significant bandwidth savings, without impacting the player quality of experience. DeepGame does not change the source code of the video encoder or the game engine, nor does it require any additional hardware or software at the client side.

Dr. Balakrishnan “Prabha” Prabhakaran

Prabha (Balakrishnan) Prabhakaran is currently a Program Director in the Human Centered Computing (HCC) program of Information and Intelligent Systems (IIS) Division of the CISE (Computer and Information Science and Engineering) Directorate of the National Science Foundation (NSF). He is a Co-Lead for Smart Health and Biomedical Research in the Era of Artificial Intelligence and Advanced Data Science (SCH, in partnership with National Institute of Health) and is a member of USA Government’s Digital Health research & Development (DHRD) Inter-agency Working Group. Prabhakaran works with the National AI (Artificial Intelligence) Research Institutes program. He also represents IIS in the Future of Work (FW-HTF), Secure and Trustworthy Computing (SaTC), US-India Research Collaboration as well as other programs such as Fairness in Artificial Intelligence.

Prabhakaran is also a Professor in the faculty of Computer Science Department, University of Texas at Dallas. Prabhakaran received the prestigious NSF CAREER Award FY 2003 for his proposal on Animation Databases. He was selected as an ACM Distinguished Scientist in 2011. Prabhakaran is an Associate Editor-in-Chief for IEEE MultiMedia. He served as an Associate Editor of IEEE Transactions on Multimedia. He is Member of the Editorial board of Multimedia Systems Journal (Springer), Multimedia Tools and Applications journal (Springer), and other multimedia systems journals. He received the Best Associate Editor for 2015, from Springer’s Multimedia Systems Journal. Prabhakaran has been serving as General and TPC (Technical Program Committee) Co-Chair for leading multimedia and health informatics conferences. He is a Member of the Executive Council of the ACM Special Interest Group on Multimedia (SIGMM). Full CV.

Overview of Balakrishnan's talk:

Title: Research Opportunities in NSF for eXtended Reality (XR)

The US National Science Foundation supports fundamental as well as use-inspired research in the broad area of eXtended Reality (XR). The CISE (Computer and Information Science and Engineering) of NSF has multiple divisions: The Information and Intelligent Systems (IIS) division funds research related to the inter-related roles of people, computers, and information. The division supports research in human-computer interaction, data science, and artificial intelligence.  IIS includes three core programs, Human-Centered Computing (HCC), Information Integration and Informatics (III), and Robust Intelligence (RI).  The division also contributes to many interdisciplinary crosscutting programs. The talk will provide an overview of research opportunities in IIS and some other interdisciplinary programs that Prabha is involved in, such as:

• Smart and Connected Health (SCH)
• Future of Work at the Human-Technology Frontier: Core Research (FW-HTF)
• Secure and Trustworthy Cyberspace (SaTC)

Prof. Ali C. Begen

Ali C. Begen is currently a computer science professor at Ozyegin University and a technical consultant in Comcast's Advanced Technology and Standards Group. Previously, he was a research and development engineer at Cisco. Begen received his PhD in electrical and computer engineering from Georgia Tech in 2006. To date, he received several academic and industry awards (including an Emmy® Award for Technology and Engineering), and was granted 30+ US patents. In 2020 and 2021, he was listed among the world's most influential scientists in the subfield of networking and telecommunications.

Overview of Ali's talk:

Title: A Master's Toolbox and Algorithms for Low-Latency Live Streaming

Today, a glass-to-glass latency of 10-30 seconds is practically achievable in live streaming and such a range is acceptable in most cases. Yet, the increasing number of cord-cutters is putting pressure on streaming providers to offer low-latency (2-10 seconds) streaming, especially for sports content. In the last few years, the streaming industry produced a number of solutions. At the high level, there are common features across all these solutions but there are also obvious as well as subtle differences in their requirements and implementations. Independent of the technology, low-latency live streaming brings up new challenges. This talk covers some of these aspects, shows examples and surveys various efforts that are underway in the streaming industry.