# Metadata Source URL:: https://nlmap-saycan.github.io/ Topics:: #ai --- # NLMap-SayCan Project page for Open-vocabulary Queryable Scene Representations for Real World Planning ## Highlights > [!quote]+ Updated on 210922_185228 > > In this paper, we develop NLMap, an > open-vocabulary and queryable scene representation to address this > problem. NLMap serves as a framework to gather and integrate > contextual information into LLM planners, allowing them to see and > query available objects in the scene before generating a > context-conditioned plan. NLMap first establishes a natural language > queryable scene representation with Visual Language models (VLMs). An > LLM based object proposal module parses instructions and proposes > involved objects to query the scene representation for object > availability and location. An LLM planner then plans with such > information about the scene. NLMap allows robots to operate without a > fixed list of objects nor executable options, enabling real robot > operation unachievable by previous methods.