Training AI to predict Myers-Briggs Personality Types From Texts

When I was in college, I used to watch Gilmore Girls, an American comedy-drama series created by Amy Sherman-Palladino, starring Lauren Graham and Alexis Bledel. I watched the 7 seasons, plus the sequel released in 2016.

Loved the mix of characters with different personalities. I was a big fan of Rory Gilmore and Logan Huntzberger!

So, I found a great dataset to play with, the Myers-Brigss 16 Personality Types that I can use to train a model and detect different types of personalities in texts and even better, test it on Gilmore Girls characters to predict their personalities!

Myers-Briggs 16 Personality Types

According to the Myers & Briggs Foundation, there are 16 personality types created by Isabel Briggs and Katharine Cook Brigss on the work of Carl Jung based on four key dimensions that could be used to categorize people:

• Introversion vs. Extraversion
• Sensing vs. Intuition
• Thinking vs. Feeling
• Judging vs. Perceiving

Below, I list the key dimensions and the 16 different personality types:

ISTJ (The Inspector)

Quiet, serious, earn success by thoroughness and dependability. Practical, matter-of-fact, realistic, and responsible. Decide logically what should be done and work toward it steadily, regardless of distractions. Take pleasure in making everything orderly and organized — their work, their home, their life. Value traditions and loyalty.

ISFJ (The Nurturer)

Quiet, friendly, responsible, and conscientious. Committed and steady in meeting their obligations. Thorough, painstaking, and accurate. Loyal, considerate, notice and remember specifics about people who are important to them, concerned with how others feel. Strive to create an orderly and harmonious environment at work and at home.

INFJ (The Counselor)

Seek meaning and connection in ideas, relationships, and material possessions. Want to understand what motivates people and are insightful about others. Conscientious and committed to their firm values. Develop a clear vision about how best to serve the common good. Organized and decisive in implementing their vision.

INTJ (The Mastermind)

Have original minds and great drive for implementing their ideas and achieving their goals. Quickly see patterns in external events and develop long-range explanatory perspectives. When committed, organize a job and carry it through. Skeptical and independent, have high standards of competence and performance — for themselves and others.

ISTP (The Craftsman)

Tolerant and flexible, quiet observers until a problem appears, then act quickly to find workable solutions. Analyze what makes things work and readily get through large amounts of data to isolate the core of practical problems. Interested in cause and effect, organize facts using logical principles, value efficiency.

ISFP (The Composer)

Quiet, friendly, sensitive, and kind. Enjoy the present moment, what’s going on around them. Like to have their own space and to work within their own time frame. Loyal and committed to their values and to people who are important to them. Dislike disagreements and conflicts, do not force their opinions or values on others.

INFP (The Idealist)

Idealistic, loyal to their values and to people who are important to them. Want an external life that is congruent with their values. Curious, quick to see possibilities, can be catalysts for implementing ideas. Seek to understand people and to help them fulfill their potential. Adaptable, flexible, and accepting unless a value is threatened.

INTP (The Thinker)

Seek to develop logical explanations for everything that interests them. Theoretical and abstract, interested more in ideas than in social interaction. Quiet, contained, flexible, and adaptable. Have an unusual ability to focus in depth to solve problems in their area of interest. Skeptical, sometimes critical, always analytical.

ESTP (The Doer)

Flexible and tolerant, they take a pragmatic approach focused on immediate results. Theories and conceptual explanations bore them — they want to act energetically to solve the problem. Focus on the here-and-now, spontaneous, enjoy each moment that they can be active with others. Enjoy material comforts and style. Learn best through doing.

ESFP (The Performer)

Outgoing, friendly, and accepting. Exuberant lovers of life, people, and material comforts. Enjoy working with others to make things happen. Bring common sense and a realistic approach to their work, and make work fun. Flexible and spontaneous, adapt readily to new people and environments. Learn best by trying a new skill with other people.

ENFP (The Champion)

Warmly enthusiastic and imaginative. See life as full of possibilities. Make connections between events and information very quickly, and confidently proceed based on the patterns they see. Want a lot of affirmation from others, and readily give appreciation and support. Spontaneous and flexible, often rely on their ability to improvise and their verbal fluency.

ENTP (The Visionary)

Quick, ingenious, stimulating, alert, and outspoken. Resourceful in solving new and challenging problems. Adept at generating conceptual possibilities and then analyzing them strategically. Good at reading other people. Bored by routine, will seldom do the same thing the same way, apt to turn to one new interest after another.

ESTJ (The Supervisor)

Practical, realistic, matter-of-fact. Decisive, quickly move to implement decisions. Organize projects and people to get things done, focus on getting results in the most efficient way possible. Take care of routine details. Have a clear set of logical standards, systematically follow them and want others to also. Forceful in implementing their plans.

ESFJ (The Provider)

Warmhearted, conscientious, and cooperative. Want harmony in their environment, work with determination to establish it. Like to work with others to complete tasks accurately and on time. Loyal, follow through even in small matters. Notice what others need in their day-by-day lives and try to provide it. Want to be appreciated for who they are and for what they contribute.

ENFJ (The Giver)

Warm, empathetic, responsive, and responsible. Highly attuned to the emotions, needs, and motivations of others. Find potential in everyone, want to help others fulfill their potential. May act as catalysts for individual and group growth. Loyal, responsive to praise and criticism. Sociable, facilitate others in a group, and provide inspiring leadership.

ENTJ (The Commander)

Frank, decisive, assume leadership readily. Quickly see illogical and inefficient procedures and policies, develop and implement comprehensive systems to solve organizational problems. Enjoy long-term planning and goal setting. Usually well informed, well read, enjoy expanding their knowledge and passing it on to others. Forceful in presenting their ideas.

Can AI be trained to identify these types of personalities from texts?

This may not be the first time someone tries this. As a matter of fact, found some papers about the implementation of this idea but different ways to tackle. One of those is proposed in the paper called “A multi-label, semi-supervised classification approach applied to personality prediction in social media” by Ana Carolina E.S. Lima and Leandro Nunes de Castro. They proposed a system trained with three machine-learning algorithms: Naïve Bayes classifier, a Support Vector Machine, and a Multilayer Perceptron neural network. The system was applied to predict the personality of Tweets taken from three datasets and got an accuracy of 83%.

To keep the same essence, I will create a model using Natural Language Processing (NLP) techniques and the text module of the fastai library because it contains all the necessary functions to define a dataset suitable for the various NLP tasks and quickly generate a model that can understand the human language.

(MBTI) Myers-Briggs Personality Type Dataset

As always, I found an interesting dataset to do this experiment: “(MBTI) Myers-Briggs Personality Type Dataset”, which contains over 8,600 rows of data, on each row is a person’s:

• Type (This person 4 letter MBTI code/type)
• A section of each of the last 50 things they have posted (Each entry separated by “|||” (3 pipe characters))

Let´s check out the data:

We don’t have null values, great.

Which class has more samples?

As you can see in the plot, ‘INFP’, ‘INFJ’, ‘INTP’ and ‘INTJ’ are the classes with more samples in the dataset.

Let’s check the most popular words in some personality types:

People with INFP personality type use these words:

Multi-Class Classification with FastAi

We have a multi-class classification problem, also called multinomial classifiers, that can distinguish between more than two classes. This will also be an NLP task, so it will be of much help to use a pre-trained state-of-the-art deep learning model and tune it to serve our purpose, you may know that this is called transfer learning. We don’t need to train a model from scratch which can be slow and intensive!

Next thing to do is to create the models: Language model and the classifier:

# Language model data

data_lm = TextLMDataBunch.from_csv(path, ‘personality.csv’)

# Classifier model data

data_clas = TextClasDataBunch.from_csv(path, ‘personality.csv’, vocab=data_lm.train_ds.vocab, bs=32)

After the text preprocessing, we save the results:

data_lm.save(‘data_lm_export.pkl’)

data_clas.save(‘data_clas_export.pkl’)

We now load what we saved before:

data_lm = load_data(path, ‘data_lm_export.pkl’)

data_clas = load_data(path, ‘data_clas_export.pkl’, bs=16)

We created a learner to start the training:

learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.5)

learn.fit_one_cycle(1, 1e-2)

We train again:

learn.unfreeze()

learn.fit_one_cycle(1, 1e-3)

And again…

learn.unfreeze()

learn.fit_one_cycle(1, slice(2e-3/100, 2e-3))

Let’s make some prediction in the language model to see what we are getting:

learn.predict(“If all these things should”, n_words=10)

‘If all these things should be done, it would be shocking or pointless to’

Let´s save the model for our classifier:

learn.save_encoder(‘ft_enc’)

Now, it’s time to create a learner object for our classifier model:

learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)

learn.load_encoder(‘ft_enc’)

If you are curious about how the data looks, let’s see the first rows:

data_clas.show_batch()

Now, it’s time to train the classifier model!

learn.fit_one_cycle(1, 1e-2)

We can unfreeze the model and fine-tune it to get a better accuracy:

learn.freeze_to(-2)

learn.fit_one_cycle(1, slice(5e-3/2., 5e-3))

learn.unfreeze()

learn.fit_one_cycle(1, slice(2e-3/100, 2e-3))

Predicting Gilmore Girls Characters’ Personality

Let´s start with Rory Gilmore:

• “Nothing excites me before 11:00”
• “I really do hate everyone today, including myself”
• “It’s Avril Lavigne’s world. We’re just living in it”
• “College is not just a crazy, wild, sleep-deprived hedonistic society”
• “My books look sad. Can books look sad?”
• “I’ll eat to that”
• “Butt-faced miscreant!”
• “No. I just take a book with me everywhere. It’s just a habit”

Picked this one:

learn.predict(“Hey, I’m not looking for social contacts. I have friends. I’m fine“)

Our model predicts that Rory Gilmore has a personality type: Introverted, Sensing, Feeling, Perceiving (ISFP). You can read the full description below:

Quiet, friendly, sensitive, and kind. Enjoy the present moment, what’s going on around them. Like to have their own space and to work within their own time frame. Loyal and committed to their values and to people who are important to them. Dislike disagreements and conflicts, do not force their opinions or values on others.

Wow! I always thought that Rory was reserved and quiet. I agree that this kind of people want to have their own space:

I think this is accurate, Rory always appreciated her friends and was loyal to them. And she was always avoiding problems, a neutral person, so, I agree that she dislikes disagreements and conflicts.

Now, let’s try the model on Logan Huntzberger:

• No, that’s what I want you to call me from now on
• It’s your choice Ace. People can live a hundred years without really living for a minute. You climb up here with me, It’s one less minute you haven’t lived
• It’ll be fun, it’ll be a thrill. something stupid, something bad for you. Just something different
• If I saw me doing this, I’d beat the crap out of my face
• Wow! So that’s what hard work feels like. Apparently, I’ve been avoiding it for a reason

Picked this line:

learn.predict(“It’ll be fun, it’ll be a thrill. something stupid, something bad for you. Just something different“)

The model says that Logan has this personality type: Extraverted, Intuitive, Feeling, Perceiving (INFP).

Warmly enthusiastic and imaginative. See life as full of possibilities. Make connections between events and information very quickly, and confidently proceed based on the patterns they see. Want a lot of affirmation from others, and readily give appreciation and support. Spontaneous and flexible, often rely on their ability to improvise and their verbal fluency.

I can’t agree more with this result! Logan was an enthusiastic character. He always was very supportive to Rory though. Being part of The Life and Death Brigade made him do many stupid and risky things. So I agree that he’s that kind of spontaneous people in the world. He’s a master of the speech, too!

Well, I’m not a psychologist and don’t plan to be. These are the insights that I got, which I think they are great!. You can try to use the same dataset and make predictions on different texts!

References