NLMap-SayCan - nlmap-saycan.github.io  ## Metadata - Author: **nlmap-saycan.github.io** - Full Title: NLMap-SayCan - Category: #articles - Tags: #ai - URL: https://nlmap-saycan.github.io/ ## Highlights - In this paper, we develop NLMap, an open-vocabulary and queryable scene representation to address this problem. NLMap serves as a framework to gather and integrate contextual information into LLM planners, allowing them to see and query available objects in the scene before generating a context-conditioned plan. NLMap first establishes a natural language queryable scene representation with Visual Language models (VLMs). An LLM based object proposal module parses instructions and proposes involved objects to query the scene representation for object availability and location. An LLM planner then plans with such information about the scene. NLMap allows robots to operate without a fixed list of objects nor executable options, enabling real robot operation unachievable by previous methods.