Following are the entities in Concierge:
- payload (p): This represents the payload received to add a new document to index.
- document (d): This represents all the metadata representing a unique document.
- line (l): This represents all the metadata for a single line within a document.
- token (t): This represents all the metadata for each token within a document.
- Message (xMsg): For a given entity, x, it provides information to identify a unique entity and a callback channel to return the unique entity.
- Process Channels (xProcessCh): For a given entity, x, the channel is used by xProcessor goroutine to consume and process the entity.
- Stores (or Data Stores): The Concierge is also responsible for storing and maintaining information regarding all the documents and lines in the system.
- Store Channels (xStoreCh): For a given entity, x, the channel is used to update the entity's store.
- Get Channels (xGetCh or xGetAllCh): These channels are used by stores to provide a mechanism to retrieve an entity using callback channels.
- done: This is a special channel that will stop all running goroutines once it is closed. We should be careful to close this channel and not send a message on it, the reason being that sending a message will only signal a single goroutine to stop. Instead, if we were to close the channel, all goroutines listening on the channel will receive message to stop.
Let's look at a few examples so that we have perfect understanding of the conventions:
- dStoreCh: This is the channel to add new documents to Document Store
- dGetCh: This is the channel to get a single document from Document Store