Elastic D&D - Update 4 - Text Note Input

Last week we talked about App Page 2 and the Account tab. If you missed it, you can check that out here!

Coding the Note Input Tab

This tab was actually the first section of code that I completed for this project! It was my main goal to create something easy-to-use for my group, even if they knew nothing about Elastic.

The tab is comprised of 3 forms that appear based on what you select in the log type select box: a general form, a form for new quests, and a form for existing quests.

The goal of all 3 forms is to get enough relevant data to form a JSON payload to send to Elastic for indexing. From there your notes are stored and you are able to search them.

NOTE:

Using forms is really nice because the data in the widgets is removed once you hit submit. This saves you from having to manually remove your note that you just typed. However, this only removes that data from the GUI, not the session state.

I used a combination of the "text_form_variable_list" list and the "clear_session_state" function to maintain a clear session state for every note. If this wasn't in here, you may have lingering data in variables and your notes wouldn't be as accurate.

Remember that "st.experimental_rerun()" is a flow control mechanism. It helps break us out of loops and if statements, ultimately getting us to the "clear_session_state" function at the bottom.

def app_page2_text():
    # displays text note form and widgets
    import json

    #list of variables to clear from session state once finished
    text_form_variable_list = ["log_type","log_session","note_taker","log_index","quest_type","quest_name","quest_finished","log_message","submitted","log_payload"]

    # displays note form widgets, creates note payload, sends payload to an Elastic index, and handles error / success / warning messages
    st.session_state["log_type"] = st.selectbox("What kind of note is this?", ["location","miscellaneous","overview","person","quest"])
    # displays note form for quest log type
    if st.session_state.log_type == "quest":
        st.session_state["quest_type"] = st.selectbox("Is this a new or existing quest?", ["New","Existing"])
        if st.session_state.quest_type == "New":
            with st.form("text_form_new_quest", clear_on_submit=True):
                st.session_state["log_session"] = st.slider("Which session is this?", 0, 250)
                st.session_state["quest_name"] = st.text_input("What is the name of the quest?")
                st.session_state["quest_finished"] = st.checkbox("Did you finish the quest?")
                st.session_state["log_message"] = st.text_input("Input note text:")
                st.session_state["submitted"] = st.form_submit_button("Upload note")
                if st.session_state.submitted == True and st.session_state.log_message is not None:
                    st.session_state["log_payload"] = json.dumps({"finished":st.session_state.quest_finished,"message":st.session_state.log_message,"name":st.session_state.quest_name,"session":st.session_state.log_session,"type":st.session_state.log_type})
                    elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                    st.experimental_rerun()
                else:
                    st.warning('Please input note text and submit')                
        else:
            quest_names = elastic_get_quests()
            with st.form("text_form_existing_quest", clear_on_submit=True):
                st.session_state["log_session"] = st.slider("Which session is this?", 0, 250)
                st.session_state["quest_name"] = st.selectbox("Which quest are you updating?", quest_names)
                st.session_state["quest_finished"] = st.checkbox("Did you finish the quest?")
                st.session_state["log_message"] = st.text_input("Input note text:")
                st.session_state["submitted"] = st.form_submit_button("Upload note")
                if st.session_state.submitted == True and st.session_state.log_message is not None:
                    # updates previous quest records to finished: true
                    if st.session_state.quest_finished == True:
                        elastic_update_quest_status(st.session_state.quest_name)
                    else:
                        pass
                    st.session_state["log_payload"] = json.dumps({"finished":st.session_state.quest_finished,"message":st.session_state.log_message,"name":st.session_state.quest_name,"session":st.session_state.log_session,"type":st.session_state.log_type})
                    elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                    st.experimental_rerun()
                else:
                    st.warning('Please input note text and submit')
    # displays note form for all other log types
    else:
        with st.form("text_form_wo_quest", clear_on_submit=True):
            st.session_state["log_session"] = st.number_input("Which session is this?", 0, 250)
            st.session_state["log_message"] = st.text_input("Input note text:")
            st.session_state["submitted"] = st.form_submit_button("Upload Note")
            if st.session_state.submitted == True and st.session_state.log_message is not None:
                st.session_state["log_payload"] = json.dumps({"message":st.session_state.log_message,"session":st.session_state.log_session,"type":st.session_state.log_type})
                elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                st.experimental_rerun()
            else:
                st.warning('Please input note text and submit')

    # clears session state
    clear_session_state(text_form_variable_list)

General Form

The general form is used for multiple log types: locations, miscellaneous information, session overviews, and people/NPCs. It has the user input a session number and type their note. This compiles a JSON object from variables in the session state and sends it to a function for indexing to Elastic.

else:
        with st.form("text_form_wo_quest", clear_on_submit=True):
            st.session_state["log_session"] = st.number_input("Which session is this?", 0, 250)
            st.session_state["log_message"] = st.text_input("Input note text:")
            st.session_state["submitted"] = st.form_submit_button("Upload Note")
            if st.session_state.submitted == True and st.session_state.log_message is not None:
                st.session_state["log_payload"] = json.dumps({"message":st.session_state.log_message,"session":st.session_state.log_session,"type":st.session_state.log_type})
                elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                st.experimental_rerun()
            else:
                st.warning('Please input note text and submit')

Elastic Index Document Function

This function is quite simple: connect to Elastic, index the JSON payload created from the form, return a success or error message, and close the connection to Elastic. No lingering open connections and very easy to re-use anywhere in the code.

def elastic_index_document(index,document):
    # sends a document to an Elastic index
    from elasticsearch import Elasticsearch

    # creates Elastic connection
    client = Elasticsearch(
        elastic_url,
        ca_certs=elastic_ca_certs,
        api_key=elastic_api_key
    )

    # sends document to index with success or failure message
    response = client.index(index=index,document=document)

    if response["result"] == "created":
        success_message("Note creation successful")
    else:
        error_message("Note creation failure")

    # close Elastic connection
    client.close()

NOTE:

I am writing about this function here, but you will see it present in both blocks of code below. Indexing information to Elastic is the purpose of this tab.

New Quest Form

The new quest form is used for just that: creating a log for a quest that does not exist in your notes yet. It has the user input a session number, the quest name (which will be used later), if the quest was finished or not, and a note about the quest. This compiles a JSON object from variables in the session state and sends it to a function for indexing to Elastic.

if st.session_state.quest_type == "New":
            with st.form("text_form_new_quest", clear_on_submit=True):
                st.session_state["log_session"] = st.slider("Which session is this?", 0, 250)
                st.session_state["quest_name"] = st.text_input("What is the name of the quest?")
                st.session_state["quest_finished"] = st.checkbox("Did you finish the quest?")
                st.session_state["log_message"] = st.text_input("Input note text:")
                st.session_state["submitted"] = st.form_submit_button("Upload note")
                if st.session_state.submitted == True and st.session_state.log_message is not None:
                    st.session_state["log_payload"] = json.dumps({"finished":st.session_state.quest_finished,"message":st.session_state.log_message,"name":st.session_state.quest_name,"session":st.session_state.log_session,"type":st.session_state.log_type})
                    elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                    st.experimental_rerun()
                else:
                    st.warning('Please input note text and submit')

Existing Quest Form

The existing quest form is for creating additional notes providing updates on quest progress. It has the user input a session number, if the quest was finished or not, and a note about the quest. This compiles a JSON object from variables in the session state and sends it to a function for indexing to Elastic.

The major difference about this form is that is also provides the user with a drop-down list of unfinished quests to choose from. This keeps our data consistent by means capitalization, punctuation, and format. This list is generated by the "elastic_get_quests" function. If you finished a quest and check the box to confirm that, another function will run to update that quest status for all previous notes as well.

else:
            quest_names = elastic_get_quests()
            with st.form("text_form_existing_quest", clear_on_submit=True):
                st.session_state["log_session"] = st.slider("Which session is this?", 0, 250)
                st.session_state["quest_name"] = st.selectbox("Which quest are you updating?", quest_names)
                st.session_state["quest_finished"] = st.checkbox("Did you finish the quest?")
                st.session_state["log_message"] = st.text_input("Input note text:")
                st.session_state["submitted"] = st.form_submit_button("Upload note")
                if st.session_state.submitted == True and st.session_state.log_message is not None:
                    # updates previous quest records to finished: true
                    if st.session_state.quest_finished == True:
                        elastic_update_quest_status(st.session_state.quest_name)
                    else:
                        pass
                    st.session_state["log_payload"] = json.dumps({"finished":st.session_state.quest_finished,"message":st.session_state.log_message,"name":st.session_state.quest_name,"session":st.session_state.log_session,"type":st.session_state.log_type})
                    elastic_index_document(st.session_state.log_index,st.session_state.log_payload)
                    st.experimental_rerun()
                else:
                    st.warning('Please input note text and submit')

Elastic Get Quests Function

This function is fairly simple as well, however, it can be confusing if you aren't familiar with Elastic query DSL. It connects to Elastic, runs a terms aggregation on unfinished quests, appends the results to list to return, and closes the connection to Elastic.

def elastic_get_quests():
    # queries Elastic for unfinished quests and returns array
    from elasticsearch import Elasticsearch

    quest_names = []

    # creates Elastic connection
    client = Elasticsearch(
        elastic_url,
        ca_certs=elastic_ca_certs,
        api_key=elastic_api_key
    )

    # gets unfinished quests
    response = client.search(index=st.session_state.log_index,size=0,query={"bool":{"must":[{"match":{"type.keyword":"quest"}}],"must_not":[{"match":{"finished":"true"}}]}},aggregations={"unfinished_quests":{"terms":{"field":"name.keyword"}}})

    for line in response["aggregations"]["unfinished_quests"]["buckets"]:
        quest_names.append(line["key"])

    return quest_names

Elastic Update Quest Status Function

This function is a bit more complex, in that it has multiple steps to work through. It connects to Elastic, queries a quest name and returns all logs for it that are marked as unfinished, updates all of those logs as finished, and closes the connection to Elastic.

def elastic_update_quest_status(quest_name):
    # queries Elastic for unfinished quests and returns array
    from elasticsearch import Elasticsearch

    # creates Elastic connection
    client = Elasticsearch(
        elastic_url,
        ca_certs=elastic_ca_certs,
        api_key=elastic_api_key
    )

    # gets unfinished quests
    query_response = client.search(index=st.session_state.log_index,size=10000,query={"bool":{"must":[{"match":{"name.keyword":quest_name}}],"must_not":[{"match":{"finished":"true"}}]}})

    for line in query_response["hits"]["hits"]:
        line_id = line["_id"]
        update_response = client.update(index="dnd-notes-corver_flickerspring",id=line_id,doc={"finished":st.session_state.quest_finished})

    # close Elastic connection
    client.close()

Related Functions

def clear_session_state(variable_list):
    # deletes variables from streamlit session state
    for variable in variable_list:
        try:
            del st.session_state[variable]
        except:
            pass

def error_message(text):
    # displays error message
    import time

    error = st.error(text)
    time.sleep(1)
    error.empty()

def success_message(text):
    # displays success message
    import time

    success = st.success(text)
    time.sleep(1)
    success.empty()

Closing Remarks

This section is definitely subject to change. As I begin work on the Virtual DM tab, the note format and fields may change; meaning I would have to adjust this whole section of code accordingly. I will definitely post updates along the way!

Next week, I will be covering the Audio Input tab, which turned out to be quite fun. I learned a new skill, as this was something I had never done before.

Check out the GitHub repo below. You can also find my Twitch account in the socials link, where I will be actively working on this during the week while interacting with whoever is hanging out!

GitHub Repo
Socials

Happy Coding,
Joe