Getting audio scores

e647b8409f8b4bd180689d4cb48ba9b9

Import required packages

[2]:

from oceanai.modules.lab.build import Run

Build

[3]:

_b5 = Run(
    lang = 'en', # Interface language
    color_simple = '#333', # Plain text color (hexadecimal code)
    color_info = '#1776D2', # The color of the text containing the information (hexadecimal code)
    color_err = '#FF0000', # Error text color (hexadecimal code)
    color_true = '#008001', # Text color containing positive information (hexadecimal code)
    bold_text = True, # Bold text
    num_to_df_display = 30, # Number of rows to display in tables
    text_runtime = 'Runtime', # Runtime text
    metadata = True # Displaying information about library
)

[2023-12-14 16:54:20] OCEANAI - personality traits: Authors: Elena Ryumina [ryumina_ev@mail.ru] Dmitry Ryumin [dl_03.03.1991@mail.ru] Alexey Karpov [karpov@iias.spb.su] Maintainers: Elena Ryumina [ryumina_ev@mail.ru] Dmitry Ryumin [dl_03.03.1991@mail.ru] Version: 1.0.0a16 License: BSD License

Getting and displaying versions of installed libraries

_b5.df_pkgs_ - DataFrame with versions of installed libraries

[4]:

_b5.libs_vers(runtime = True, run = True)

	Package	Version
1	TensorFlow	2.15.0
2	Keras	2.15.0
3	OpenCV	4.8.1
4	MediaPipe	0.9.0
5	NumPy	1.26.2
6	SciPy	1.11.4
7	Pandas	2.1.3
8	Scikit-learn	1.3.2
9	OpenSmile	2.5.0
10	Librosa	0.10.1
11	AudioRead	3.0.1
12	IPython	8.18.1
13	PyMediaInfo	6.1.0
14	Requests	2.31.0
15	JupyterLab	4.0.9
16	LIWC	0.5.0
17	Transformers	4.36.0
18	Sentencepiece	0.1.99
19	Torch	2.0.1+cpu
20	Torchaudio	2.0.2+cpu

— Runtime: 0.005 sec. —

Formation of the neural network architecture of the model for obtaining scores by hand-crafted features

_b5.audio_model_hc_ - Neural network model tf.keras.Model for obtaining scores by hand-crafted features

[5]:

res_load_audio_model_hc = _b5.load_audio_model_hc(
    show_summary = False, # Displaying the formed neural network architecture of the model
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:20] Formation of the neural network architecture of the model for obtaining scores by hand-crafted features (audio modality) …

— Runtime: 0.335 sec. —

Downloading the weights of the neural network model to obtain scores by hand-crafted features

_b5.audio_model_hc_ - Neural network model tf.keras.Model for obtaining scores by hand-crafted features

[6]:

# Core settings
_b5.path_to_save_ = './models' # Directory to save the file
_b5.chunk_size_ = 2000000 # File download size from network in 1 step

url = _b5.weights_for_big5_['audio']['fi']['hc']['sberdisk']

res_load_audio_model_weights_hc = _b5.load_audio_model_weights_hc(
    url = url, # Full path to the file with weights of the neural network model
    force_reload = True, # Forced download of a file with weights of a neural network model from the network
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:21] Downloading the weights of the neural network model to obtain scores by hand-crafted features (audio modality) …

[2023-12-14 16:54:21] File download “weights_2022-05-05_11-27-55.h5” (100.0%) …

— Runtime: 0.323 sec. —

Formation of the neural network architecture of the model for obtaining scores by deep features

_b5.audio_model_nn_ - Neural network model tf.keras.Model for obtaining scores by deep features

[7]:

res_load_audio_model_nn = _b5.load_audio_model_nn(
    show_summary = False, # Displaying the formed neural network architecture of the model
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:21] Formation of a neural network architecture for obtaining scores by deep features (audio modality) …

— Runtime: 0.212 sec. —

Downloading the weights of the neural network model to obtain scores by deep features

_b5.audio_model_nn_ - Neural network model tf.keras.Model for obtaining scores by deep features

[8]:

# Core settings
_b5.path_to_save_ = './models' # Directory to save the file
_b5.chunk_size_ = 2000000 # File download size from network in 1 step

url = _b5.weights_for_big5_['audio']['fi']['nn']['sberdisk']

res_load_audio_model_weights_nn = _b5.load_audio_model_weights_nn(
    url = url, # Full path to the file with weights of the neural network model
    force_reload = False, # Forced download of a file with weights of a neural network model from the network
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:21] Downloading the weights of the neural network model to obtain scores for deep features (audio modality) …

[2023-12-14 16:54:22] File download “weights_2022-05-03_07-46-14.h5”

— Runtime: 0.416 sec. —

Formation of neural network architectures of models for obtaining the personality traits scores

_b5.audio_models_b5_ - Neural network models tf.keras.Model for obtaining the personality traits scores

[9]:

res_load_audio_models_b5 = _b5.load_audio_models_b5(
    show_summary = False, # Displaying the formed neural network architecture of the model
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:22] Formation of neural network architectures of models for obtaining personality traits scores (audio modality) …

— Runtime: 0.067 sec. —

Downloading the weights of neural network models to obtain the personality traits scores

_b5.audio_models_b5_ - Neural network models tf.keras.Model for obtaining the personality traits scores

[10]:

# Core settings
_b5.path_to_save_ = './models' # Directory to save the file
_b5.chunk_size_ = 2000000 # File download size from network in 1 step

url_openness = _b5.weights_for_big5_['audio']['fi']['b5']['openness']['sberdisk']
url_conscientiousness = _b5.weights_for_big5_['audio']['fi']['b5']['conscientiousness']['sberdisk']
url_extraversion = _b5.weights_for_big5_['audio']['fi']['b5']['extraversion']['sberdisk']
url_agreeableness = _b5.weights_for_big5_['audio']['fi']['b5']['agreeableness']['sberdisk']
url_non_neuroticism = _b5.weights_for_big5_['audio']['fi']['b5']['non_neuroticism']['sberdisk']

res_load_audio_models_weights_b5 = _b5.load_audio_models_weights_b5(
    url_openness = url_openness, # Openness
    url_conscientiousness = url_conscientiousness, # Conscientiousness
    url_extraversion = url_extraversion, # Extraversion
    url_agreeableness = url_agreeableness, # Agreeableness
    url_non_neuroticism = url_non_neuroticism, # Non-Neuroticism
    force_reload = True, # Forced download of a file with weights of a neural network model from the network
    out = True, # Display
    runtime = True, # Runtime count
    run = True # Run blocking
)

[2023-12-14 16:54:22] Downloading the weights of neural network models to obtain the personality traits scores (audio modality) …

[2023-12-14 16:54:22] File download “weights_2022-06-15_16-16-20.h5” (100.0%) … Openness

[2023-12-14 16:54:22] File download “weights_2022-06-15_16-21-57.h5” (100.0%) … Conscientiousness

[2023-12-14 16:54:22] File download “weights_2022-06-15_16-26-41.h5” (100.0%) … Extraversion

[2023-12-14 16:54:22] File download “weights_2022-06-15_16-32-51.h5” (100.0%) … Agreeableness

[2023-12-14 16:54:22] File download “weights_2022-06-15_16-37-46.h5” (100.0%) … Non-Neuroticism

— Runtime: 0.807 sec. —

Getting scores (audio modality)

_b5.df_files_ - DataFrame with data

_b5.df_accuracy_ - DataFrame with accuracy

[11]:

# Core settings
_b5.path_to_dataset_ = 'E:/Databases/FirstImpressionsV2/test' # Dataset directory
# Directories not included in the set
_b5.ignore_dirs_ = []
# Key names for DataFrame dataset
_b5.keys_dataset_ = ['Path', 'Openness', 'Conscientiousness', 'Extraversion', 'Agreeableness', 'Non-Neuroticism']
_b5.ext_ = ['.mp4'] # Search file extensions
_b5.path_to_logs_ = './logs' # Directory for saving LOG files

# Full path to the file containing the ground truth scores for the accuracy calculation
url_accuracy = _b5.true_traits_['fi']['sberdisk']

res_get_audio_union_predictions = _b5.get_audio_union_predictions(
    depth = 2,         # Hierarchy depth for receiving audio and video data
    recursive = False, # Recursive data search
    sr = 44100,        # Sampling frequency
    window = 2,        # Signal segment window size (in seconds)
    step = 1,          # Signal segment window shift step (in seconds)
    accuracy = True,   # Accuracy
    url_accuracy = url_accuracy,
    logs = True,      # If necessary, generate a LOG file
    out = True,        # Display
    runtime = True,    # Runtime count
    run = True         # Run blocking
)

[2023-12-14 17:59:22] Getting scores and accuracy calculation (audio modality) …

2000 from 2000 (100.0%) … test80_25_Q4wOgixh7E.004.mp4 …

	Path	Openness	Conscientiousness	Extraversion	Agreeableness	Non-Neuroticism
ID
1	E:\Databases\FirstImpressionsV2\test\test80_01...	0.603529	0.556223	0.526545	0.579621	0.547629
2	E:\Databases\FirstImpressionsV2\test\test80_01...	0.568246	0.465263	0.460744	0.541769	0.511338
3	E:\Databases\FirstImpressionsV2\test\test80_01...	0.546209	0.603946	0.469445	0.589493	0.545716
4	E:\Databases\FirstImpressionsV2\test\test80_01...	0.691056	0.623856	0.628851	0.614669	0.645813
5	E:\Databases\FirstImpressionsV2\test\test80_01...	0.690808	0.589734	0.636104	0.606598	0.63479
6	E:\Databases\FirstImpressionsV2\test\test80_01...	0.65728	0.681336	0.571412	0.596052	0.623451
7	E:\Databases\FirstImpressionsV2\test\test80_01...	0.453781	0.438842	0.376464	0.520368	0.438252
8	E:\Databases\FirstImpressionsV2\test\test80_01...	0.558594	0.598366	0.452183	0.618858	0.571653
9	E:\Databases\FirstImpressionsV2\test\test80_01...	0.529081	0.502482	0.426603	0.488263	0.443719
10	E:\Databases\FirstImpressionsV2\test\test80_01...	0.537279	0.508283	0.438888	0.579794	0.512117
11	E:\Databases\FirstImpressionsV2\test\test80_01...	0.512779	0.447352	0.422968	0.559107	0.491406
12	E:\Databases\FirstImpressionsV2\test\test80_01...	0.447102	0.451113	0.364429	0.513031	0.414412
13	E:\Databases\FirstImpressionsV2\test\test80_01...	0.368372	0.391985	0.274865	0.42951	0.307666
14	E:\Databases\FirstImpressionsV2\test\test80_01...	0.582539	0.432871	0.412363	0.441974	0.462192
15	E:\Databases\FirstImpressionsV2\test\test80_01...	0.627705	0.801831	0.528622	0.692623	0.691908
16	E:\Databases\FirstImpressionsV2\test\test80_01...	0.708798	0.654007	0.640547	0.632052	0.669044
17	E:\Databases\FirstImpressionsV2\test\test80_01...	0.583968	0.644164	0.50463	0.633507	0.59208
18	E:\Databases\FirstImpressionsV2\test\test80_01...	0.550836	0.539624	0.468092	0.594872	0.544016
19	E:\Databases\FirstImpressionsV2\test\test80_01...	0.626745	0.563271	0.556561	0.561901	0.549236
20	E:\Databases\FirstImpressionsV2\test\test80_01...	0.593014	0.421482	0.504798	0.534224	0.532807
21	E:\Databases\FirstImpressionsV2\test\test80_01...	0.545921	0.479671	0.465769	0.571302	0.518793
22	E:\Databases\FirstImpressionsV2\test\test80_01...	0.548432	0.480831	0.453319	0.52774	0.47759
23	E:\Databases\FirstImpressionsV2\test\test80_01...	0.486083	0.467779	0.396113	0.444633	0.399402
24	E:\Databases\FirstImpressionsV2\test\test80_01...	0.558323	0.537912	0.474172	0.563599	0.52937
25	E:\Databases\FirstImpressionsV2\test\test80_01...	0.473017	0.542138	0.370228	0.550093	0.467068
26	E:\Databases\FirstImpressionsV2\test\test80_01...	0.530967	0.460241	0.410618	0.507322	0.450027
27	E:\Databases\FirstImpressionsV2\test\test80_01...	0.61807	0.506396	0.572248	0.574811	0.563796
28	E:\Databases\FirstImpressionsV2\test\test80_01...	0.64703	0.577771	0.565869	0.575279	0.60631
29	E:\Databases\FirstImpressionsV2\test\test80_01...	0.571473	0.529536	0.48662	0.535691	0.529022
30	E:\Databases\FirstImpressionsV2\test\test80_01...	0.655007	0.606712	0.592804	0.570543	0.600349

[2023-12-14 17:59:22] Trait-wise accuracy …

	Openness	Conscientiousness	Extraversion	Agreeableness	Non-Neuroticism	Mean
Metrics
MAE	0.0916	0.0925	0.0932	0.0918	0.094	0.0926
Accuracy	0.9084	0.9075	0.9068	0.9082	0.906	0.9074

[2023-12-14 17:59:22] Mean absolute error: 0.0926, Accuracy: 0.9074 …

Log files saved successfully …

— Runtime: 3899.26 sec. —