.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "mmwave_PC/mmwave_PC_har_tutorial.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_mmwave_PC_mmwave_PC_har_tutorial.py: Tutorial for Human Activity Recognition ============================================================== .. GENERATED FROM PYTHON SOURCE LINES 8-10 !/usr/bin/env python coding: utf-8 .. GENERATED FROM PYTHON SOURCE LINES 10-24 .. code-block:: Python # ------------------------ # In[1]: import yaml import torch import torch.nn as nn from tqdm import tqdm import os .. GENERATED FROM PYTHON SOURCE LINES 25-27 Dataset with radHAR: ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 29-32 radHAR dataset is designed to use mmWave PC data collected by IWR1443Ti to predict the actions of the users. There are totally 5 actions in the dataset: ['boxing','jack','jump','squats','walk'] In the library, we provide a dataloader to use mmWave PC data , converted into voxel image, and predict these actions. .. GENERATED FROM PYTHON SOURCE LINES 35-37 Load the data ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 39-40 In[2]: .. GENERATED FROM PYTHON SOURCE LINES 40-46 .. code-block:: Python from pysensing.mmwave.PC.dataset.har import load_har_dataset # The path contains the radHAR dataset train_dataset, test_dataset = load_har_dataset("radHAR") .. rst-class:: sphx-glr-script-out .. code-block:: none Try to download radHAR dateset in /home/kemove/yyz/av-gihub/tutorials/mmwave_PC_source/radHAR Downloading radHAR to /home/kemove/yyz/av-gihub/tutorials/mmwave_PC_source/radHAR.zip... Downloading: 0%| | 0.00/115M [00:00 .. GENERATED FROM PYTHON SOURCE LINES 67-69 Create model ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 71-72 raHAR utilizes MLP-based model as a baseline har method. From model.har, we can import desired har model designed for mmWave PC. The model parameter for har_MLP reimplemented for radHAR is as follows: .. GENERATED FROM PYTHON SOURCE LINES 74-75 In[4]: .. GENERATED FROM PYTHON SOURCE LINES 75-81 .. code-block:: Python from pysensing.mmwave.PC.model.har import har_MLP model = har_MLP(dataset="radHAR", num_classes=5) print(model) .. rst-class:: sphx-glr-script-out .. code-block:: none har_MLP( (fc): Sequential( (0): Linear(in_features=614400, out_features=64, bias=True) (1): ReLU() (2): Dropout(p=0.5, inplace=False) (3): Linear(in_features=64, out_features=64, bias=True) (4): ReLU() (5): Dropout(p=0.5, inplace=False) (6): Linear(in_features=64, out_features=64, bias=True) (7): ReLU() (8): Dropout(p=0.5, inplace=False) ) (classifier): Sequential( (0): Linear(in_features=64, out_features=5, bias=True) (1): Softmax(dim=-1) ) ) .. GENERATED FROM PYTHON SOURCE LINES 82-84 Model Train ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 86-87 pysensing library support quick training of model with the following steps. The training interface incorporates pytorch loss functions, optimizers and dataloaders to facilate training. An example is provided for how to define the aforemetioned terms. .. GENERATED FROM PYTHON SOURCE LINES 89-90 # In[5]: .. GENERATED FROM PYTHON SOURCE LINES 90-108 .. code-block:: Python # Create pytorch dataloaders train_loader = torch.utils.data.DataLoader(train_dataset, shuffle=True, batch_size=128, num_workers=16) test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=128, shuffle=False, num_workers=16) # Define pytorch loss function as criterion criterion = nn.CrossEntropyLoss() # Define pytorch optimizer for training optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) # GPU acceleration with cuda device = torch.device("cuda" if torch.cuda.is_available() else "cpu") .. GENERATED FROM PYTHON SOURCE LINES 109-110 A quick training using har_train. The resulted model parameters will be saved into "train_{num_epochs}.pth". .. GENERATED FROM PYTHON SOURCE LINES 112-113 In[6]: .. GENERATED FROM PYTHON SOURCE LINES 113-119 .. code-block:: Python # Pysensing training interface from pysensing.mmwave.PC.inference.har import har_train # har_train(model, train_loader, num_epochs=1, optimizer=optimizer, criterion=criterion, device=device) .. GENERATED FROM PYTHON SOURCE LINES 120-122 Model inference ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 125-127 Load the pretrained model, e.g. from https://pysensing.oss-ap-southeast-1.aliyuncs.com/pretrain/mmwave_pc/HAR/radHAR_MLP.pth , and perform human action recognition! .. GENERATED FROM PYTHON SOURCE LINES 129-130 In[7]: .. GENERATED FROM PYTHON SOURCE LINES 130-136 .. code-block:: Python # load pretrained model from pysensing.mmwave.PC.inference import load_pretrain model = load_pretrain(model, "radHAR", "har_MLP").to(device) model.eval() .. rst-class:: sphx-glr-script-out .. code-block:: none Use pretrained model! har_MLP( (fc): Sequential( (0): Linear(in_features=614400, out_features=64, bias=True) (1): ReLU() (2): Dropout(p=0.5, inplace=False) (3): Linear(in_features=64, out_features=64, bias=True) (4): ReLU() (5): Dropout(p=0.5, inplace=False) (6): Linear(in_features=64, out_features=64, bias=True) (7): ReLU() (8): Dropout(p=0.5, inplace=False) ) (classifier): Sequential( (0): Linear(in_features=64, out_features=5, bias=True) (1): Softmax(dim=-1) ) ) .. GENERATED FROM PYTHON SOURCE LINES 137-138 Test the model on testing dataset. .. GENERATED FROM PYTHON SOURCE LINES 140-141 In[8]: .. GENERATED FROM PYTHON SOURCE LINES 141-144 .. code-block:: Python from pysensing.mmwave.PC.inference.har import har_test # har_test(model, test_loader, criterion=criterion, device=device) .. GENERATED FROM PYTHON SOURCE LINES 145-146 Model inference on sample and deep feature embedding of input modality in HAR task. .. GENERATED FROM PYTHON SOURCE LINES 148-149 In[9]: .. GENERATED FROM PYTHON SOURCE LINES 149-166 .. code-block:: Python idx = 5 pc, label= test_dataset.__getitem__(idx) print(pc.shape) pc = torch.tensor(pc).unsqueeze(0).float().to(device) predicted_result = model(pc) print("The predicted gesture is {}, while the ground truth is {}".format(label,torch.argmax(predicted_result).cpu())) # Deep feature embedding from pysensing.mmwave.PC.inference.embedding import embedding emb = embedding(input = pc, model=model, dataset_name = "radHAR", model_name = "har_MLP", device=device) print("The shape of feature embedding is: ", emb.shape) .. rst-class:: sphx-glr-script-out .. code-block:: none (60, 10, 32, 32) The predicted gesture is 0, while the ground truth is 0 The shape of feature embedding is: torch.Size([1, 64]) .. rst-class:: sphx-glr-timing **Total running time of the script:** (1 minutes 45.125 seconds) .. _sphx_glr_download_mmwave_PC_mmwave_PC_har_tutorial.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: mmwave_PC_har_tutorial.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: mmwave_PC_har_tutorial.py ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_