Research

SYNTHETIC IMAGE DATASETS OFFER UNMATCHED ADVANTAGES FOR DESIGNING AND EVALUATING DEEP NEURAL NETWORKS: THEY MAKE IT POSSIBLE TO (I) RENDER AS MANY DATA SAMPLES AS NEEDED, (II) PRECISELY CONTROL EACH SCENE AND YIELD GRANULAR GROUND TRUTH LABELS (AND CAPTIONS), (III) PRECISELY CONTROL DISTRIBUTION SHIFTS BETWEEN TRAINING AND TESTING TO ISOLATE VARIABLES OF INTEREST FOR SOUND EXPERIMENTATION.

THERE HAS BEEN AN INCREASING RESEARCH INTEREST IN DEVELOPING SPECIALIZED DIALOGUE SYSTEMS THAT CAN OFFER MENTAL HEALTH SUPPORT.

DIFFERENT FROM THE PREVIOUS SELF-KNOWLEDGE DISTILLATION, THIS STAGE FINETUNES THE STUDENT'S HEAD WITH ONLY 20% TRAINING TIME AS A PLUG-AND-PLAY TRAINING STRATEGY.

WE, FOR THE FIRST TIME, PROPOSE AN IMAGE RETRIEVAL PARADIGM LEVERAGING GLOBAL FEATURE ONLY TO ENABLE ACCURATE AND LIGHTWEIGHT IMAGE RETRIEVAL FOR BOTH COARSE RETRIEVAL AND RERANKING, THUS THE NAME - SUPERGLOBAL.

THIS EVOLUTION REQUIRES A COMBINATION OF BOTH TRADITIONAL METHODS (SUCH AS TERM-BASED SPARSE RETRIEVAL METHODS WITH RAPID RESPONSE) AND MODERN NEURAL ARCHITECTURES (SUCH AS LANGUAGE MODELS WITH POWERFUL LANGUAGE UNDERSTANDING CAPACITY).

JOINTLY PROCESSING INFORMATION FROM MULTIPLE SENSORS IS CRUCIAL TO ACHIEVING ACCURATE AND ROBUST PERCEPTION FOR RELIABLE AUTONOMOUS DRIVING SYSTEMS.

WE PRESENT THE CONTENT DEFORMATION FIELD F AS A NEW TYPE OF VIDEO REPRESENTATION, WHICH CONSISTS OF A CANONICAL CONTENT FIELD AGGREGATING THE STATIC CONTENTS IN THE ENTIRE VIDEO AND A TEMPORAL DEFORMATION FIELD RECORDING THE TRANSFORMATIONS FROM THE CANONICAL IMAGE (I. E., RENDERED FROM THE CANONICAL CONTENT FIELD) TO EACH INDIVIDUAL FRAME ALONG THE TIME AXIS. GIVEN A TARGET VIDEO, THESE TWO FIELDS ARE JOINTLY OPTIMIZED TO RECONSTRUCT IT THROUGH A CAREFULLY TAILORED RENDERING PIPELINE. WE ADVISEDLY INTRODUCE SOME REGULARIZATIONS INTO THE OPTIMIZATION PROCESS, URGING THE CANONICAL CONTENT FIELD TO INHERIT SEMANTICS (E. G., THE OBJECT SHAPE) FROM THE VIDEO. WITH SUCH A DESIGN, F NATURALLY SUPPORTS LIFTING IMAGE ALGORITHMS FOR VIDEO PROCESSING, IN THE SENSE THAT ONE CAN APPLY AN IMAGE ALGORITHM TO THE CANONICAL IMAGE AND EFFORTLESSLY PROPAGATE THE OUTCOMES TO THE ENTIRE VIDEO WITH THE AID OF THE TEMPORAL DEFORMATION FIELD. WE EXPERIMENTALLY SHOW THAT F IS ABLE TO LIFT IMAGE-TO-IMAGE TRANSLATION TO VIDEO-TO-VIDEO TRANSLATION AND LIFT KEYPOINT DETECTION TO KEYPOINT TRACKING WITHOUT ANY TRAINING. MORE IMPORTANTLY, THANKS TO OUR LIFTING STRATEGY THAT DEPLOYS THE ALGORITHMS ON ONLY ONE IMAGE, WE ACHIEVE SUPERIOR CROSS-FRAME CONSISTENCY IN PROCESSED VIDEOS COMPARED TO EXISTING VIDEO-TO-VIDEO TRANSLATION APPROACHES, AND EVEN MANAGE TO TRACK NON-RIGID OBJECTS LIKE WATER AND SMOG. PROJECT PAGE CAN BE FOUND AT IMAGE-TO-IMAGE TRANSLATION KEYPOINT DETECTION +1

IN THIS WORK, WE PROPOSE A NOVEL TRAINING MECHANISM TERMED SEGPROMPT THAT USES CATEGORY INFORMATION TO IMPROVE THE MODEL'S CLASS-AGNOSTIC SEGMENTATION ABILITY FOR BOTH KNOWN AND UNKNOWN CATEGORIES.

MESH IS EXTRACTED FROM THE SIGNED DISTANCE FUNCTION (SDF) NETWORK FOR THE SURFACE, AND COLOR FOR EACH SURFACE VERTEX IS DRAWN FROM THE GLOBAL COLOR NETWORK.

WE INTRODUCE MUAVIC, A MULTILINGUAL AUDIO-VISUAL CORPUS FOR ROBUST SPEECH RECOGNITION AND ROBUST SPEECH-TO-TEXT TRANSLATION PROVIDING 1200 S OF AUDIO-VISUAL SPEECH IN 9 LANGUAGES.

RECENTLY, REMARKABLE PROGRESS HAS BEEN MADE IN AUTOMATED TASK-SOLVING THROUGH THE USE OF MULTI-AGENT DRIVEN BY LARGE LANGUAGE MODELS (LLMS).

RADIANCE FIELD METHODS HAVE RECENTLY REVOLUTIONIZED NOVEL-VIEW SYNTHESIS OF SCENES CAPTURED WITH MULTIPLE PHOTOS OR VIDEOS.

LARGE PRE-TRAINED GENERATION MODELS, SUCH AS OPENAI X, CAN GENERATE SYNTAX- AND FUNCTION-CORRECT , MAKING THE CODING OF PROGRAMMERS MORE PRODUCTIVE AND OUR PURSUIT OF ARTIFICIAL GENERAL INTELLIGENCE CLOSER.

TO SHOWCASE THE POWER OF THE PROPOSED APPROACH, WE GENERATE DATASETS WITH RICH DENSE PIXEL-WISE LABELS FOR A WIDE RANGE OF DOWNSTREAM TASKS, INCLUDING SEMANTIC SEGMENTATION, INSTANCE SEGMENTATION, AND DEPTH ESTIMATION.

WE PROPOSE A NOVEL FRAMEWORK CIPHERCHAT TO SYSTEMATICALLY EXAMINE THE GENERALIZABILITY OF SAFETY ALIGNMENT TO NON-NATURAL LANGUAGES -- CIPHERS.

WE FIRST PRESENT TOOLBENCH, AN INSTRUCTION-TUNING DATASET FOR TOOL USE, WHICH IS CREATED AUTOMATICALLY USING CHATGPT.

MOREOVER, OUR MODEL EVEN OUTPERFORMS THE LARGEST CLOSED LLMS, ANTHROPIC'S CLAUDE AND GOOGLE'S BARD, ON HUMANEVAL AND HUMANEVAL+.

HOWEVER, RLHF IS A COMPLEX AND OFTEN UNSTABLE PROCEDURE, FIRST FITTING A REWARD MODEL THAT REFLECTS THE HUMAN PREFERENCES, AND THEN FINE-TUNING THE LARGE UNSUPERVISED LM USING REINFORCEMENT LEARNING TO MAXIMIZE THIS ESTIMATED REWARD WITHOUT DRIFTING TOO FAR FROM THE ORIGINAL MODEL.

WE PRESENT A NEAR REAL-TIME METHOD FOR 6-DOF TRACKING OF AN UNKNOWN OBJECT FROM A MONOCULAR RGBD VIDEO SEQUENCE, WHILE SIMULTANEOUSLY PERFORMING NEURAL 3D RECONSTRUCTION OF THE OBJECT.

WE DEMONSTRATE FAN ON A REAL-WORLD ROBOTIC SYSTEM (A MICRO AERIAL VEHICLE) AND REPORT ITS ABILITY TO SEAMLESSLY FOLLOW THE OBJECTS OF INTEREST IN A REAL-TIME CONTROL LOOP.