Audio

class oceanai.modules.lab.audio.AudioMessages(lang: str = 'ru', color_simple: str = '#666', color_info: str = '#1776D2', color_err: str = '#FF0000', color_true: str = '#008001', bold_text: bool = True, text_runtime: str = '', num_to_df_display: int = 30)

Bases: Download

Class for messages

Parameters:

• lang (str) – See lang

• color_simple (str) – See color_simple

• color_info (str) – See color_info

• color_err (str) – See color_err

• color_true (str) – See color_true

• bold_text (bool) – See bold_text

• num_to_df_display (int) – See num_to_df_display

• text_runtime (str) – See text_runtime

class oceanai.modules.lab.audio.Audio(lang: str = 'ru', color_simple: str = '#666', color_info: str = '#1776D2', color_err: str = '#FF0000', color_true: str = '#008001', bold_text: bool = True, text_runtime: str = '', num_to_df_display: int = 30)

Bases: AudioMessages

Audio processing class

Parameters:

• lang (str) – See lang

• color_simple (str) – See color_simple

• color_info (str) – See color_info

• color_err (str) – See color_err

• color_true (str) – See color_true

• bold_text (bool) – See bold_text

• num_to_df_display (int) – See num_to_df_display

• text_runtime (str) – See text_runtime

__concat_pred(pred_hc: ndarray, pred_melspectrogram: ndarray, out: bool = True) → List[ndarray | None]

Concatenation of scores by hand-crafted and deep features

Note

private method

Parameters:

• pred_hc (np.ndarray) – Scores based on had-crafted features

• pred_melspectrogram (np.ndarray) – Scores based on deep features

• out (bool) – Display

Returns:

Concatenated scores by hand-crafted and deep features

Return type:

List[Optional[np.ndarray]]

Examples

True – 1 –

In [1]:

import numpy as np
from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

arr_hc = np.array([
    [0.64113516, 0.6217892, 0.54451424, 0.6144415, 0.59334993],
    [0.6652424, 0.63606125, 0.572305, 0.63169795, 0.612515]
])

arr_melspectrogram = np.array([
    [0.56030345, 0.7488746, 0.44648764, 0.59893465, 0.5701077],
    [0.5900006, 0.7652722, 0.4795154, 0.6409055, 0.6088242]
])

audio._Audio__concat_pred(
    pred_hc = arr_hc,
    pred_melspectrogram = arr_melspectrogram,
    out = True
)

[1]:

[
    array([
            0.64113516, 0.6652424, 0.65318878, 0.65318878, 0.65318878,
            0.65318878, 0.65318878, 0.65318878, 0.65318878, 0.65318878,
            0.65318878, 0.65318878, 0.65318878, 0.65318878, 0.65318878,
            0.65318878, 0.56030345, 0.5900006, 0.57515202, 0.57515202,
            0.57515202, 0.57515202, 0.57515202, 0.57515202, 0.57515202,
            0.57515202, 0.57515202, 0.57515202, 0.57515202, 0.57515202,
            0.57515202, 0.57515202
    ]),
    array([
            0.6217892, 0.63606125, 0.62892523, 0.62892523, 0.62892523,
            0.62892523, 0.62892523, 0.62892523, 0.62892523, 0.62892523,
            0.62892523, 0.62892523, 0.62892523, 0.62892523, 0.62892523,
            0.62892523, 0.7488746, 0.7652722, 0.7570734, 0.7570734,
            0.7570734, 0.7570734, 0.7570734, 0.7570734, 0.7570734,
            0.7570734, 0.7570734, 0.7570734, 0.7570734, 0.7570734,
            0.7570734, 0.7570734
    ]),
    array([
            0.54451424, 0.572305, 0.55840962, 0.55840962, 0.55840962,
            0.55840962, 0.55840962, 0.55840962, 0.55840962, 0.55840962,
            0.55840962, 0.55840962, 0.55840962, 0.55840962, 0.55840962,
            0.55840962, 0.44648764, 0.4795154, 0.46300152, 0.46300152,
            0.46300152, 0.46300152, 0.46300152, 0.46300152, 0.46300152,
            0.46300152, 0.46300152, 0.46300152, 0.46300152, 0.46300152,
            0.46300152, 0.46300152
    ]),
    array([
            0.6144415, 0.63169795, 0.62306972, 0.62306972, 0.62306972,
            0.62306972, 0.62306972, 0.62306972, 0.62306972, 0.62306972,
            0.62306972, 0.62306972, 0.62306972, 0.62306972, 0.62306972,
            0.62306972, 0.59893465, 0.6409055, 0.61992008, 0.61992008,
            0.61992008, 0.61992008, 0.61992008, 0.61992008, 0.61992008,
            0.61992008, 0.61992008, 0.61992008, 0.61992008, 0.61992008,
            0.61992008, 0.61992008
    ]),
    array([
            0.59334993, 0.612515, 0.60293247, 0.60293247, 0.60293247,
            0.60293247, 0.60293247, 0.60293247, 0.60293247, 0.60293247,
            0.60293247, 0.60293247, 0.60293247, 0.60293247, 0.60293247,
            0.60293247, 0.5701077, 0.6088242, 0.58946595, 0.58946595,
            0.58946595, 0.58946595, 0.58946595, 0.58946595, 0.58946595,
            0.58946595, 0.58946595, 0.58946595, 0.58946595, 0.58946595,
            0.58946595, 0.58946595
    ])
]

Error – 1 –

In [2]:

import numpy as np
from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

arr_hc = np.array([
    [0.64113516, 0.6217892, 0.54451424, 0.6144415],
    [0.6652424, 0.63606125, 0.572305, 0.63169795, 0.612515]
])

arr_melspectrogram = np.array([
    [0.56030345, 0.7488746, 0.44648764, 0.59893465, 0.5701077],
    [0.5900006, 0.7652722, 0.4795154, 0.6409055, 0.6088242]
])

audio._Audio__concat_pred(
    pred_hc = arr_hc,
    pred_melspectrogram = arr_melspectrogram,
    out = True
)

[3]:

[2022-10-20 22:33:31] Something went wrong ... concatenation of scores by hand-crafted and deep features was not performed (audio modality) ...

[]

__load_audio_model_b5(show_summary: bool = False, out: bool = True) → Module | None

Formation of the neural network architecture of the model to obtain the personality traits scores

Note

private method

Parameters:

• show_summary (bool) – Displaying the formed neural network architecture of the model

• out (bool) – Display

Returns:

None если неверные типы или значения аргументов, в обратном случае нейросетевая модель nn.Module для получения результата оценки персонального качества

Return type:

Optional[nn.Module]

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio._Audio__load_audio_model_b5(
    show_summary = True, out = True
)

[1]:

audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
)
audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
)

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio._Audio__load_audio_model_b5(
    show_summary = True, out = []
)

[3]:

[2022-10-17 10:53:03] Invalid argument types or values in "Audio.__load_audio_model_b5" ...

__load_model_weights(url: str, force_reload: bool = True, info_text: str = '', out: bool = True, runtime: bool = True, run: bool = True) → bool

Downloading the weights of the neural network model

Note

private method

Parameters:

• url (str) – Full path to the file with weights of the neural network model

• force_reload (bool) – Forced download of a file with weights of a neural network model from the network

• info_text (str) – Text for informational message

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the weights of the neural network model are downloaded, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

audio._Audio__load_model_weights(
    url = 'https://download.sberdisk.ru/download/file/400635799?token=MMRrak8fMsyzxLE&filename=weights_2022-05-05_11-27-55.h5',
    force_reload = True,
    info_text = 'Downloading the weights of the neural network model',
    out = True, runtime = True, run = True
)

[1]:

[2022-10-17 12:21:48] Downloading the weights of the neural network model

[2022-10-17 12:21:48] File download "weights_2022-05-05_11-27-55.h5" (100.0%) ...

--- Runtime: 0.439 sec. ---

True

– 2 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

audio._Audio__load_model_weights(
    url = './models/weights_2022-05-05_11-27-55.h5',
    force_reload = True,
    info_text = 'Downloading the weights of the neural network model',
    out = True, runtime = True, run = True
)

[2]:

[2022-10-17 12:21:50] Downloading the weights of the neural network model

--- Runtime: 0.002 sec. ---

True

Error – 1 –

In [3]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

audio._Audio__load_model_weights(
    url = 'https://download.sberdisk.ru/download/file/400635799?token=MMRrak8fMsyzxLE&filename=weights_2022-05-05_11-27-55.h5',
    force_reload = True, info_text = '',
    out = True, runtime = True, run = True
)

[3]:

[2022-10-17 12:21:57] Invalid argument types or values in "Audio.__load_model_weights" ...

False

__norm_pred(pred_data: ndarray, len_spec: int = 16, out: bool = True) → ndarray

Normalization of scores by hand-crafted and deep features

Note

private method

Parameters:

• pred_data (np.ndarray) – Scores

• len_spec (int) – The maximum size of the scores vector

• out (bool) – Display

Returns:

Normalized scores by hand-crafted and deep features

Return type:

np.ndarray

Examples

True – 1 –

In [1]:

import numpy as np
from oceanai.modules.lab.audio import Audio

audio = Audio()

arr = np.array([
    [0.64113516, 0.6217892, 0.54451424, 0.6144415, 0.59334993],
    [0.6652424, 0.63606125, 0.572305, 0.63169795, 0.612515]
])

audio._Audio__norm_pred(
    pred_data = arr,
    len_spec = 4,
    out = True
)

[1]:

array([
    [0.64113516, 0.6217892 , 0.54451424, 0.6144415 , 0.59334993],
    [0.6652424 , 0.63606125, 0.572305  , 0.63169795, 0.612515],
    [0.65318878, 0.62892523, 0.55840962, 0.62306972, 0.60293247],
    [0.65318878, 0.62892523, 0.55840962, 0.62306972, 0.60293247]
])

Error – 1 –

In [2]:

import numpy as np
from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

arr = np.array([])

audio._Audio__norm_pred(
    pred_data = arr,
    len_spec = 4,
    out = True
)

[3]:

[2022-10-20 22:03:17] Invalid argument types or values in "Audio.__norm_pred" ...

array([], dtype=float64)

__smile() → Smile

Extracting OpenSmile features

Note

private method

Returns:

Extracted OpenSmile features

Return type:

opensmile.core.smile.Smile

Example

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio._Audio__smile()

[1]:

{
    '$opensmile.core.smile.Smile': {
        'feature_set': 'eGeMAPSv02',
        'feature_level': 'LowLevelDescriptors',
        'options': {},
        'sampling_rate': None,
        'channels': [0],
        'mixdown': False,
        'resample': False
    }
}

_get_acoustic_features(path: str, sr: int = 44100, window: int | float = 2.0, step: int | float = 1.0, last: bool = False, out: bool = True, runtime: bool = True, run: bool = True) → Tuple[List[ndarray | None], List[ndarray | None]]

Extracting features from an acoustic signal (without clearing the message output history in a Jupyter cell)

Note

protected method

Parameters:

• path (str) – Path to the audio or video file

• sr (int) – Sampling frequency

• window (Union[int, float]) – Signal segment window size (in seconds)

• step (Union[int, float]) – Signal segment window shift step (in seconds)

• last (bool) – Replacing the last message

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

Tuple with two lists: 1. List with hand-crafted features 2. List with mel-spectrograms

Return type:

Tuple[List[Optional[np.ndarray]], List[Optional[np.ndarray]]]

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

sr = 44100
path = '/Users/dl/GitHub/oceanai/oceanai/dataset/test80_01/glgfB3vFewc.004.mp4'

hc_features, melspectrogram_features = audio._get_acoustic_features(
    path = path, sr = sr,
    window = 2, step = 1,
    last = False, out = True,
    runtime = True, run = True
)

[1]:

[2022-10-19 14:58:19] Extraction of features (hand-crafted and mel-spectrograms) from an acoustic signal ...

[2022-10-19 14:58:20] Statistics of the features extracted from the acoustic signal:
    Total number of segments with:
    1. hand-crafted features: 12
    2. mel-spectrogram log: 12
    Dimension of the matrix of hand-crafted features of one segment: 196 ✕ 25
    Dimension of the tensor with log mel-spectrograms of one segment: 224 ✕ 224 ✕ 3

--- Runtime: 1.273 sec. ---

Errors – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

sr = 44100
path = '/Users/dl/GitHub/oceanai/oceanai/dataset/test80_01/glgfB3vFewc.004.mp4'

hc_features, melspectrogram_features = audio._get_acoustic_features(
    path = 1, sr = sr,
    window = 2, step = 1,
    last = False, out = True,
    runtime = True, run = True
)

[2]:

[2022-10-19 15:33:04] Invalid argument types or values in "Audio._get_acoustic_features" ...

– 2 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

sr = 44100
path = '/Users/dl/GitHub/oceanai/oceanai/dataset/test80_01/glgfB3vFewc.004.mp4'

hc_features, melspectrogram_features = audio._get_acoustic_features(
    path = path, sr = sr,
    window = 0.04, step = 1,
    last = False, out = True,
    runtime = True, run = True
)

[2]:

[2022-10-19 15:34:38] Extraction of features (hand-crafted and mel-spectrograms) from an acoustic signal ...

[2022-10-19 15:34:38] Something went wrong ... the size (0.04) of the signal segment window is too small ...

    File: /Users/dl/GitHub/oceanai/oceanai/modules/lab/audio.py
    Line: 863
    Method: _get_acoustic_features
    Error type: IsSmallWindowSizeError

--- Runtime: 0.049 sec. ---

property audio_model_hc_: Module | None

Получение нейросетевой модели nn.Module для получения оценок по экспертным признакам

Returns:

Нейросетевая модель nn.Module или None

Return type:

Optional[nn.Module]

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_model_hc(
    show_summary = False, out = True,
    runtime = True, run = True
)

audio.audio_model_hc_

[1]:

[2024-10-06 22:49:58] Формирование нейросетевой архитектуры модели для получения оценок по экспертным признакам (аудио модальность) ...

--- Время выполнения: 0.011 сек. ---

audio_model_hc(
    (lstm1): LSTM(25, 64, batch_first=True)
    (dropout1): Dropout(p=0.2, inplace=False)
    (lstm2): LSTM(64, 128, batch_first=True)
    (dropout2): Dropout(p=0.2, inplace=False)
    (fc): Linear(in_features=128, out_features=5, bias=True)
)

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.audio_model_hc_

[2]:

property audio_model_nn_: Module | None

Получение нейросетевой модели nn.Module для получения оценок по нейросетевым признакам

Returns:

Нейросетевая модель nn.Module или None

Return type:

Optional[nn.Module]

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_model_nn(
    show_summary = False, out = True,
    runtime = True, run = True
)

audio.audio_model_nn_

[1]:

[2024-10-06 22:57:46] Формирование нейросетевой архитектуры для получения оценок по нейросетевым признакам (аудио модальность) ...

--- Время выполнения: 1.59 сек. ---

audio_model_nn(
(vgg): VGG(
    (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
    (classifier): Identity()
)
(flatten): Flatten(start_dim=1, end_dim=-1)
(fc1): Linear(in_features=25088, out_features=512, bias=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(fc2): Linear(in_features=512, out_features=256, bias=True)
(fc3): Linear(in_features=256, out_features=5, bias=True)
)

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.audio_model_nn_

[2]:

property audio_models_b5_: Dict[str, Module | None]

Получение нейросетевых моделей nn.Module для получения результатов оценки персональных качеств

Returns:

Словарь с нейросетевыми моделями nn.Module

Return type:

Dict

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_models_b5(
    show_summary = False, out = True,
    runtime = True, run = True
)

audio.audio_models_b5_

[1]:

[2024-10-06 22:58:27] Формирование нейросетевых архитектур моделей для получения результатов оценки персональных качеств (аудио модальность) ...

--- Время выполнения: 0.002 сек. ---

{'openness': audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
),
'conscientiousness': audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
),
'extraversion': audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
),
'agreeableness': audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
),
'non-neuroticism': audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
)}

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.audio_models_b5_

[2]:

{
    'openness': None,
    'conscientiousness': None,
    'extraversion': None,
    'agreeableness': None,
    'non_neuroticism': None
}

get_acoustic_features(path: str, sr: int = 44100, window: int | float = 2.0, step: int | float = 1.0, out: bool = True, runtime: bool = True, run: bool = True) → Tuple[List[ndarray | None], List[ndarray | None]]

Extracting features from an acoustic signal

Parameters:

• path (str) – Path to the audio or video file

• sr (int) – Sampling frequency

• window (Union[int, float]) – Signal segment window size (in seconds)

• step (Union[int, float]) – Signal segment window shift step (in seconds)

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

Tuple with two lists: 1. List with hand-crafted features 2. List with mel-spectrograms

Return type:

Tuple[List[Optional[np.ndarray]], List[Optional[np.ndarray]]]

Example

get_audio_union_predictions(depth: int = 1, recursive: bool = False, sr: int = 44100, window: int | float = 2.0, step: int | float = 1.0, accuracy=True, url_accuracy: str = '', logs: bool = True, out: bool = True, runtime: bool = True, run: bool = True) → bool

Get audio scores

Parameters:

• depth (int) – Hierarchy depth for getting data

• recursive (bool) – Recursive data search

• sr (int) – Sampling frequency

• window (Union[int, float]) – Signal segment window size (in seconds)

• step (Union[int, float]) – Signal segment window shift step (in seconds)

• accuracy (bool) – Accuracy calculation

• url_accuracy (str) – Full path to the file with ground truth scores for calculating accuracy

• logs (bool) – If necessary, generate a LOG file

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if scores are successfully received, otherwise False

Return type:

bool

Example

load_audio_model_hc(show_summary: bool = False, out: bool = True, runtime: bool = True, run: bool = True) → bool

Formation of the neural network architecture of the model for obtaining scores by hand-crafted features

Parameters:

• show_summary (bool) – Displaying the formed neural network architecture of the model

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the neural network architecture of the model is formed, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.load_audio_model_hc(
    show_summary = False, out = True,
    runtime = True, run = True
)

[1]:

[2022-10-17 13:16:23] Formation of the neural network architecture of the model for obtaining scores by hand-crafted features (audio modality) ...

--- Runtime: 0.364 sec. ---

True

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.load_audio_model_hc(
    show_summary = 1, out = True,
    runtime = True, run = True
)

[2]:

[2022-10-17 13:20:04] Invalid argument types or values in "Audio.load_audio_model_hc" ...

False

load_audio_model_nn(show_summary: bool = False, out: bool = True, runtime: bool = True, run: bool = True) → bool

Formation of a neural network architecture for obtaining scores by deep features

Parameters:

• show_summary (bool) – Displaying the formed neural network architecture of the model

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the neural network architecture of the model is formed, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio()

audio.load_audio_model_nn(
    show_summary = True, out = True,
    runtime = True, run = True
)

[1]:

[2024-10-06 23:01:20] Формирование нейросетевой архитектуры для получения оценок по нейросетевым признакам (аудио модальность) ...

audio_model_nn(
(vgg): VGG(
    (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
    (classifier): Identity()
)
(flatten): Flatten(start_dim=1, end_dim=-1)
(fc1): Linear(in_features=25088, out_features=512, bias=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(fc2): Linear(in_features=512, out_features=256, bias=True)
(fc3): Linear(in_features=256, out_features=5, bias=True)
)
--- Время выполнения: 1.958 сек. ---

True

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.load_audio_model_nn(
    show_summary = 1, out = True,
    runtime = True, run = True
)

[2]:

[2022-10-17 13:25:40] Invalid argument types or values in "Audio.load_audio_model_nn" ...

False

load_audio_model_weights_hc(url: str, force_reload: bool = True, out: bool = True, runtime: bool = True, run: bool = True) → bool

Downloading the weights of the neural network model to obtain scores by hand-crafted features

Parameters:

• url (str) – Full path to the file with weights of the neural network model

• force_reload (bool) – Forced download of a file with weights of a neural network model from the network

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the weights of the neural network model are downloaded, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_model_hc(
    show_summary = False, out = True,
    runtime = True, run = True
)

[1]:

[2022-10-17 14:24:28] Formation of the neural network architecture of the model for obtaining scores by hand-crafted features (audio modality) ...

--- Runtime: 0.398 sec. ---

True

In [2]:

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url = audio.weights_for_big5_['audio']['fi']['hc']['googledisk']

audio.load_audio_model_weights_hc(
    url = url,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[2]:

[2024-10-06 23:05:53] Загрузка весов нейросетевой модели для получения оценок по экспертным признакам (аудио модальность) ...

[2024-10-06 23:05:56] Загрузка файла "weights_2022-05-05_11-27-55.pth" 100.0% ...

--- Время выполнения: 3.078 сек. ---

True

Error – 1 –

In [3]:

from oceanai.modules.lab.audio import Audio

audio = Audio()

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url = audio.weights_for_big5_['audio']['fi']['hc']['googledisk']

audio.load_audio_model_weights_hc(
    url = url,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[3]:

[2024-10-06 23:07:25] Загрузка весов нейросетевой модели для получения оценок по экспертным признакам (аудио модальность) ...

[2024-10-06 23:07:28] Загрузка файла "weights_2022-05-05_11-27-55.pth" 100.0% ...

[2024-10-06 23:07:28] Что-то пошло не так ... нейросетевая архитектура модели для получения оценок по экспертным признакам не сформирована (аудио модальность) ...

--- Время выполнения: 2.911 сек. ---

False

load_audio_model_weights_nn(url: str, force_reload: bool = True, out: bool = True, runtime: bool = True, run: bool = True) → bool

Downloading the weights of the neural network model to obtain scores for deep features

Parameters:

• url (str) – Full path to the file with weights of the neural network model

• force_reload (bool) – Forced download of a file with weights of a neural network model from the network

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the weights of the neural network model are downloaded, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_model_nn(
    show_summary = False, out = True,
    runtime = True, run = True
)

[1]:

[2022-10-17 15:47:20] Formation of a neural network architecture for obtaining scores by deep features (audio modality) ...

--- Runtime: 0.419 sec. ---

True

In [2]:

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url = audio.weights_for_big5_['audio']['fi']['nn']['googledisk']

audio.load_audio_model_weights_nn(
    url = url,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[2]:

[2024-10-06 23:22:33] Загрузка весов нейросетевой модели для получения оценок по нейросетевым признакам (аудио модальность) ...

[2024-10-06 23:22:39] Загрузка файла "weights_2022-05-03_07-46-14.pth" 100.0% ...

--- Время выполнения: 6.454 сек. ---

True

Error – 1 –

In [3]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url = audio.weights_for_big5_['audio']['nn']['sberdisk']

audio.load_audio_model_weights_nn(
    url = url,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[3]:

[2024-10-06 23:23:37] Загрузка весов нейросетевой модели для получения оценок по нейросетевым признакам (аудио модальность) ...

[2024-10-06 23:23:43] Загрузка файла "weights_2022-05-03_07-46-14.pth" 100.0% ...

[2024-10-06 23:23:43] Что-то пошло не так ... нейросетевая архитектура модели для получения оценок по нейросетевым признакам не сформирована (аудио модальность) ...

--- Время выполнения: 5.639 сек. ---

False

load_audio_models_b5(show_summary: bool = False, out: bool = True, runtime: bool = True, run: bool = True) → bool

Formation of neural network architectures of models for obtaining the personality traits scores

Parameters:

• show_summary (bool) – Displaying the last generated neural network architecture of models

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the neural network architectures of the model are formed, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.load_audio_models_b5(
    show_summary = True, out = True,
    runtime = True, run = True
)

[1]:

[2024-10-06 23:21:04] Формирование нейросетевых архитектур моделей для получения результатов оценки персональных качеств (аудио модальность) ...

audio_model_b5(
    (fc): Linear(in_features=32, out_features=1, bias=True)
    (sigmoid): Sigmoid()
)
--- Время выполнения: 0.003 сек. ---

True

Error – 1 –

In [2]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.load_audio_models_b5(
    show_summary = 1, out = True,
    runtime = True, run = True
)

[2]:

[2022-10-18 13:47:36] Invalid argument types or values in "Audio.load_audio_models_b5" ...

False

load_audio_models_weights_b5(url_openness: str, url_conscientiousness: str, url_extraversion: str, url_agreeableness: str, url_non_neuroticism: str, force_reload: bool = True, out: bool = True, runtime: bool = True, run: bool = True) → bool

Downloading the weights of neural network models to obtain the personality traits scores

Parameters:

• url_openness (str) – Full path to the file with the weights of the neural network model (openness)

• url_conscientiousness (str) – Full path to the file with the weights of the neural network model (conscientiousness)

• url_extraversion (str) – Full path to the file with the weights of the neural network model (extraversion)

• url_agreeableness (str) – Full path to the file with the weights of the neural network model (agreeableness)

• url_non_neuroticism (str) – Full path to the file with the weights of the neural network model (non-neuroticism)

• force_reload (bool) – Forced download of files with weights of neural network models from the network

• out (bool) – Display

• runtime (bool) – Runtime count

• run (bool) – Run blocking

Returns:

True if the weights of the neural network models are downloaded, otherwise False

Return type:

bool

Examples

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')

audio.load_audio_models_b5(
    show_summary = False, out = True,
    runtime = True, run = True
)

[1]:

[2022-10-18 22:40:05] Formation of neural network architectures of models for obtaining the personality traits scores (audio modality) ...

--- Runtime: 0.163 sec. ---

True

In [2]:

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url_openness = audio.weights_for_big5_['audio']['fi']['b5']['openness']['googledisk']
url_conscientiousness = audio.weights_for_big5_['audio']['fi']['b5']['conscientiousness']['googledisk']
url_extraversion = audio.weights_for_big5_['audio']['fi']['b5']['extraversion']['googledisk']
url_agreeableness = audio.weights_for_big5_['audio']['fi']['b5']['agreeableness']['googledisk']
url_non_neuroticism = audio.weights_for_big5_['audio']['fi']['b5']['non_neuroticism']['googledisk']

audio.load_audio_models_weights_b5(
    url_openness = url_openness,
    url_conscientiousness = url_conscientiousness,
    url_extraversion = url_extraversion,
    url_agreeableness = url_agreeableness,
    url_non_neuroticism = url_non_neuroticism,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[2]:

[2024-10-06 23:15:39] Загрузка весов нейросетевых моделей для получения результатов оценки персональных качеств (аудио модальность) ...

[2024-10-06 23:15:42] Загрузка файла "weights_2022-06-15_16-16-20.pth" 100.0% ... Открытость опыту

[2024-10-06 23:15:45] Загрузка файла "weights_2022-06-15_16-21-57.pth" 100.0% ... Добросовестность

[2024-10-06 23:15:47] Загрузка файла "weights_2022-06-15_16-26-41.pth" 100.0% ... Экстраверсия

[2024-10-06 23:15:49] Загрузка файла "weights_2022-06-15_16-32-51.pth" 100.0% ... Доброжелательность

[2024-10-06 23:15:52] Загрузка файла "weights_2022-06-15_16-37-46.pth" 100.0% ... Эмоциональная стабильность

--- Время выполнения: 12.466 сек. ---

True

Error – 1 –

In [3]:

from oceanai.modules.lab.audio import Audio

audio = Audio()

audio.path_to_save_ = './models'
audio.chunk_size_ = 2000000

url_openness = audio.weights_for_big5_['audio']['fi']['b5']['openness']['googledisk']
url_conscientiousness = audio.weights_for_big5_['audio']['fi']['b5']['conscientiousness']['googledisk']
url_extraversion = audio.weights_for_big5_['audio']['fi']['b5']['extraversion']['googledisk']
url_agreeableness = audio.weights_for_big5_['audio']['fi']['b5']['agreeableness']['googledisk']
url_non_neuroticism = audio.weights_for_big5_['audio']['fi']['b5']['non_neuroticism']['googledisk']

audio.load_audio_models_weights_b5(
    url_openness = url_openness,
    url_conscientiousness = url_conscientiousness,
    url_extraversion = url_extraversion,
    url_agreeableness = url_agreeableness,
    url_non_neuroticism = url_non_neuroticism,
    force_reload = True,
    out = True,
    runtime = True,
    run = True
)

[3]:

[2024-10-06 23:17:35] Загрузка весов нейросетевых моделей для получения результатов оценки персональных качеств (аудио модальность) ...

[2024-10-06 23:17:37] Загрузка файла "weights_2022-06-15_16-16-20.pth" 100.0% ...

[2024-10-06 23:17:37] Что-то пошло не так ... не удалось загрузить веса нейросетевой модели ... Открытость опыту

    Файл: /Users/dl/@DmitryRyumin/Python/envs/OCEANAI/lib/python3.9/site-packages/oceanai/modules/lab/audio.py
    Линия: 2284
    Метод: load_audio_models_weights_b5
    Тип ошибки: AttributeError

[2024-10-06 23:17:40] Загрузка файла "weights_2022-06-15_16-21-57.pth" 100.0% ...

[2024-10-06 23:17:40] Что-то пошло не так ... не удалось загрузить веса нейросетевой модели ... Добросовестность

    Файл: /Users/dl/@DmitryRyumin/Python/envs/OCEANAI/lib/python3.9/site-packages/oceanai/modules/lab/audio.py
    Линия: 2284
    Метод: load_audio_models_weights_b5
    Тип ошибки: AttributeError

[2024-10-06 23:17:42] Загрузка файла "weights_2022-06-15_16-26-41.pth" 100.0% ...

[2024-10-06 23:17:42] Что-то пошло не так ... не удалось загрузить веса нейросетевой модели ... Экстраверсия

    Файл: /Users/dl/@DmitryRyumin/Python/envs/OCEANAI/lib/python3.9/site-packages/oceanai/modules/lab/audio.py
    Линия: 2284
    Метод: load_audio_models_weights_b5
    Тип ошибки: AttributeError

[2024-10-06 23:17:45] Загрузка файла "weights_2022-06-15_16-32-51.pth" 100.0% ...

[2024-10-06 23:17:45] Что-то пошло не так ... не удалось загрузить веса нейросетевой модели ... Доброжелательность

    Файл: /Users/dl/@DmitryRyumin/Python/envs/OCEANAI/lib/python3.9/site-packages/oceanai/modules/lab/audio.py
    Линия: 2284
    Метод: load_audio_models_weights_b5
    Тип ошибки: AttributeError

[2024-10-06 23:17:47] Загрузка файла "weights_2022-06-15_16-37-46.pth" 100.0% ...

[2024-10-06 23:17:47] Что-то пошло не так ... не удалось загрузить веса нейросетевой модели ... Эмоциональная стабильность

    Файл: /Users/dl/@DmitryRyumin/Python/envs/OCEANAI/lib/python3.9/site-packages/oceanai/modules/lab/audio.py
    Линия: 2284
    Метод: load_audio_models_weights_b5
    Тип ошибки: AttributeError

--- Время выполнения: 12.562 сек. ---

False

property smile_: Smile

Getting OpenSmile functions

Returns:

Extracted OpenSmile features

Return type:

opensmile.core.smile.Smile

Example

True – 1 –

In [1]:

from oceanai.modules.lab.audio import Audio

audio = Audio(lang = 'en')
audio.smile_

[1]:

{
    '$opensmile.core.smile.Smile': {
        'feature_set': 'eGeMAPSv02',
        'feature_level': 'LowLevelDescriptors',
        'options': {},
        'sampling_rate': None,
        'channels': [0],
        'mixdown': False,
        'resample': False
    }
}