# Metadata
Source URL:: https://nlmap-saycan.github.io/
Topics:: #ai
---
# NLMap-SayCan
Project page for Open-vocabulary Queryable Scene Representations for Real World Planning
## Highlights
> [!quote]+ Updated on 210922_185228
>
> In this paper, we develop NLMap, an
> open-vocabulary and queryable scene representation to address this
> problem. NLMap serves as a framework to gather and integrate
> contextual information into LLM planners, allowing them to see and
> query available objects in the scene before generating a
> context-conditioned plan. NLMap first establishes a natural language
> queryable scene representation with Visual Language models (VLMs). An
> LLM based object proposal module parses instructions and proposes
> involved objects to query the scene representation for object
> availability and location. An LLM planner then plans with such
> information about the scene. NLMap allows robots to operate without a
> fixed list of objects nor executable options, enabling real robot
> operation unachievable by previous methods.