使用移動(dòng)傳感器產(chǎn)生的原始數(shù)據(jù)來(lái)識(shí)別人類(lèi)活動(dòng)

• 下樓
• 上樓
• 跑步
• 坐著
• 站立
• 步行

概述

導(dǎo)入庫(kù)

from pandas import read_csv, unique
import numpy as np
from scipy.interpolate import interp1dfrom scipy.stats import mode
from sklearn.preprocessing import LabelEncoderfrom sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay
from tensorflow import stackfrom tensorflow.keras.utils import to_categoricalfrom keras.models import Sequentialfrom keras.layers import Dense, GlobalAveragePooling1D, BatchNormalization, MaxPool1D, Reshape, Activationfrom keras.layers import Conv1D, LSTMfrom keras.callbacks import ModelCheckpoint, EarlyStoppingimport matplotlib.pyplot as plt%matplotlib inline
import warningswarnings.filterwarnings("ignore")

數(shù)據(jù)集加載和可視化

def read_data(filepath):  df = read_csv(filepath, header=None, names=['user-id',                                              'activity',                                              'timestamp',                                              'X',                                              'Y',                                              'Z'])  ## removing ';' from last column and converting it to float  df['Z'].replace(regex=True, inplace=True, to_replace=r';', value=r'')  df['Z'] = df['Z'].apply(convert_to_float)  return df
def convert_to_float(x):  try:      return np.float64(x)  except:      return np.nan
df = read_data('Dataset/WISDM_ar_v1.1/WISDM_ar_v1.1_raw.txt')df

plt.figure(figsize=(15, 5))
plt.xlabel('Activity Type')plt.ylabel('Training examples')df['activity'].value_counts().plot(kind='bar',                                title='Training examples by Activity Types')plt.show()
plt.figure(figsize=(15, 5))plt.xlabel('User')plt.ylabel('Training examples')df['user-id'].value_counts().plot(kind='bar',                                title='Training examples by user')plt.show()

def axis_plot(ax, x, y, title):  ax.plot(x, y, 'r')  ax.set_title(title)  ax.xaxis.set_visible(False)  ax.set_ylim([min(y) - np.std(y), max(y) + np.std(y)])  ax.set_xlim([min(x), max(x)])  ax.grid(True)
for activity in df['activity'].unique():  limit = df[df['activity'] == activity][:180]  fig, (ax0, ax1, ax2) = plt.subplots(nrows=3, sharex=True, figsize=(15, 10))  axis_plot(ax0, limit['timestamp'], limit['X'], 'x-axis')  axis_plot(ax1, limit['timestamp'], limit['Y'], 'y-axis')  axis_plot(ax2, limit['timestamp'], limit['Z'], 'z-axis')  plt.subplots_adjust(hspace=0.2)  fig.suptitle(activity)  plt.subplots_adjust(top=0.9)  plt.show()

數(shù)據(jù)預(yù)處理

• 標(biāo)簽編碼
• 線(xiàn)性插值
• 數(shù)據(jù)分割
• 歸一化
• 時(shí)間序列分割
• 獨(dú)熱編碼

• Downstairs [0]
• Jogging [1]
• Sitting [2]
• Standing [3]
• Upstairs [4]
• Walking [5]

label_encode = LabelEncoder()df['activityEncode'] = label_encode.fit_transform(df['activity'].values.ravel())df

interpolation_fn = interp1d(df['activityEncode'] ,df['Z'], kind='linear')null_list = df[df['Z'].isnull()].index.tolist()for i in null_list:  y = df['activityEncode'][i]  value = interpolation_fn(y)  df['Z']=df['Z'].fillna(value)  print(value)

df_test = df[df['user-id'] > 27]

df_train = df[df['user-id'] <= 27]

df_train['X'] = (df_train['X']-df_train['X'].min())/(df_train['X'].max()-df_train['X'].min())df_train['Y'] = (df_train['Y']-df_train['Y'].min())/(df_train['Y'].max()-df_train['Y'].min())df_train['Z'] = (df_train['Z']-df_train['Z'].min())/(df_train['Z'].max()-df_train['Z'].min())df_train

def segments(df, time_steps, step, label_name):  N_FEATURES = 3  segments = []  labels = []  for i in range(0, len(df) - time_steps, step):      xs = df['X'].values[i:i+time_steps]      ys = df['Y'].values[i:i+time_steps]      zs = df['Z'].values[i:i+time_steps]
      label = mode(df[label_name][i:i+time_steps])[0][0]      segments.append([xs, ys, zs])      labels.append(label)
  reshaped_segments = np.asarray(segments, dtype=np.float32).reshape(-1, time_steps, N_FEATURES)  labels = np.asarray(labels)
  return reshaped_segments, labels
TIME_PERIOD = 80STEP_DISTANCE = 40LABEL = 'activityEncode'x_train, y_train = segments(df_train, TIME_PERIOD, STEP_DISTANCE, LABEL)

這樣，x_train和y_train形狀變?yōu)椋?/span>

print('x_train shape:', x_train.shape)print('Training samples:', x_train.shape[0])print('y_train shape:', y_train.shape)
x_train shape: (20334, 80, 3)Training samples: 20334y_train shape: (20334,)

這里還存儲(chǔ)了一些后面用到的數(shù)據(jù)：時(shí)間段（time_period），傳感器數(shù)（sensors）和類(lèi)（num_classes）的數(shù)量。

time_period, sensors = x_train.shape[1], x_train.shape[2]num_classes = label_encode.classes_.sizeprint(list(label_encode.classes_))
['Downstairs', 'Jogging', 'Sitting', 'Standing', 'Upstairs', 'Walking']

最后需要使用Reshape將其轉(zhuǎn)換為列表，作為keras的輸入：

input_shape = time_period * sensorsx_train = x_train.reshape(x_train.shape[0], input_shape)print("Input Shape: ", input_shape)print("Input Data Shape: ", x_train.shape)
Input Shape: 240Input Data Shape: (20334, 240)

最后需要將所有數(shù)據(jù)轉(zhuǎn)換為float32。

x_train = x_train.astype('float32')y_train = y_train.astype('float32')

獨(dú)熱編碼 這是數(shù)據(jù)預(yù)處理的最后一步，我們將通過(guò)編碼標(biāo)簽并將其存儲(chǔ)到y(tǒng)_train_hot中來(lái)執(zhí)行。

y_train_hot = to_categorical(y_train, num_classes)print("y_train shape: ", y_train_hot.shape)y_train shape: (20334, 6)

模型

?

我們使用的模型是一個(gè)由8層組成的序列模型。模型前兩層由LSTM組成，每個(gè)LSTM具有32個(gè)神經(jīng)元，使用的激活函數(shù)為Relu。然后是用于提取空間特征的卷積層。 在兩層的連接處需要改變LSTM輸出維度，因?yàn)檩敵鼍哂?個(gè)維度（樣本數(shù)，時(shí)間步長(zhǎng)，輸入維度），而CNN則需要4維輸入（樣本數(shù)，1，時(shí)間步長(zhǎng)，輸入）。 第一個(gè)CNN層具有64個(gè)神經(jīng)元，另一個(gè)神經(jīng)元有128個(gè)神經(jīng)元。在第一和第二CNN層之間，我們有一個(gè)最大池層來(lái)執(zhí)行下采樣操作。然后是全局平均池（GAP）層將多D特征映射轉(zhuǎn)換為1-D特征向量，因?yàn)樵诖藢又胁恍枰獏?shù)，所以會(huì)減少全局模型參數(shù)。然后是BN層，該層有助于模型的收斂性。 最后一層是模型的輸出層，該輸出層只是具有SoftMax分類(lèi)器層的6個(gè)神經(jīng)元的完全連接的層，該層表示當(dāng)前類(lèi)的概率。

model = Sequential()model.add(LSTM(32, return_sequences=True, input_shape=(input_shape,1), activation='relu'))model.add(LSTM(32,return_sequences=True, activation='relu'))model.add(Reshape((1, 240, 32)))model.add(Conv1D(filters=64,kernel_size=2, activation='relu', strides=2))model.add(Reshape((120, 64)))model.add(MaxPool1D(pool_size=4, padding='same'))model.add(Conv1D(filters=192, kernel_size=2, activation='relu', strides=1))model.add(Reshape((29, 192)))model.add(GlobalAveragePooling1D())model.add(BatchNormalization(epsilon=1e-06))model.add(Dense(6))model.add(Activation('softmax'))
print(model.summary())

訓(xùn)練和結(jié)果

經(jīng)過(guò)訓(xùn)練，模型給出了98.02%的準(zhǔn)確率和0.0058的損失。訓(xùn)練F1得分為0.96。

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])history = model.fit(x_train,                  y_train_hot,                  batch_size= 192,                  epochs=100                  )

?

可視化訓(xùn)練的準(zhǔn)確性和損失變化圖。

plt.figure(figsize=(6, 4))plt.plot(history.history['accuracy'], 'r', label='Accuracy of training data')plt.plot(history.history['loss'], 'r--', label='Loss of training data')plt.title('Model Accuracy and Loss')plt.ylabel('Accuracy and Loss')plt.xlabel('Training Epoch')plt.ylim(0)plt.legend()plt.show()
y_pred_train = model.predict(x_train)max_y_pred_train = np.argmax(y_pred_train, axis=1)print(classification_report(y_train, max_y_pred_train))

?

在測(cè)試數(shù)據(jù)集上測(cè)試它，但在通過(guò)測(cè)試集之前，需要對(duì)測(cè)試集進(jìn)行相同的預(yù)處理。

df_test['X'] = (df_test['X']-df_test['X'].min())/(df_test['X'].max()-df_test['X'].min())df_test['Y'] = (df_test['Y']-df_test['Y'].min())/(df_test['Y'].max()-df_test['Y'].min())df_test['Z'] = (df_test['Z']-df_test['Z'].min())/(df_test['Z'].max()-df_test['Z'].min())x_test, y_test = segments(df_test,                        TIME_PERIOD,                        STEP_DISTANCE,                        LABEL)
x_test = x_test.reshape(x_test.shape[0], input_shape)x_test = x_test.astype('float32')y_test = y_test.astype('float32')y_test = to_categorical(y_test, num_classes)

在評(píng)估我們的測(cè)試數(shù)據(jù)集后，得到了89.14%的準(zhǔn)確率和0.4647的損失。F1測(cè)試得分為0.89。

score = model.evaluate(x_test, y_test)print("Accuracy:", score[1])print("Loss:", score[0])

?

下面繪制混淆矩陣更好地理解對(duì)測(cè)試數(shù)據(jù)集的預(yù)測(cè)。

predictions = model.predict(x_test)predictions = np.argmax(predictions, axis=1)y_test_pred = np.argmax(y_test, axis=1)cm = confusion_matrix(y_test_pred, predictions)cm_disp = ConfusionMatrixDisplay(confusion_matrix= cm)cm_disp.plot()plt.show()

?還可以在測(cè)試數(shù)據(jù)集上評(píng)估的模型的分類(lèi)報(bào)告。

print(classification_report(y_test_pred, predictions))

總結(jié)

LSTM-CNN模型的性能比任何其他機(jī)器學(xué)習(xí)模型要好得多。本文的代碼可以在GitHub上找到。 https://github.com/Tanny1810/Human-Activity-Recognition-LSTM-CNN 您可以嘗試自己實(shí)現(xiàn)它，通過(guò)優(yōu)化模型來(lái)提高F1分?jǐn)?shù)。 另：這個(gè)模型是來(lái)自于Xia Kun, Huang Jianguang, and Hanyu Wang在IEEE期刊上發(fā)表的論文LSTM-CNN Architecture for Human Activity Recognition。 https://ieeexplore.ieee.org/abstract/document/9043535 審核編輯 ：李倩