失眠网 > DeepLearning | Zero shot learning 零样本学习AWA2 图像数据集预处理

DeepLearning | Zero shot learning 零样本学习AWA2 图像数据集预处理

时间：2021-07-22 23:13:10

因为有打算想要写一组关于零样本学习算法的博客，需要用到AWA2数据集作为demo演示

之前想只展示算法部分的代码就好了，但是如果只展示算法部分的代码可能不方便初学者复现，所以这里把我数据预处理的方法也说一下，博客的最后会给一个处理好的数据下载地址，之后的博客都会利用该博客的方法作为数据预处理

我会对AWA2数据集做一个详细的介绍，对数据集有一个好的理解本身也有助于算法的学习和实现

AWA2 图像数据集下载地址：http://cvml.ist.ac.at/AwA2/

数据集比较大有13个G，下载可能得花点时间

这里还有几篇零样本算法的介绍和复现：

DeepLearning | Semantic Autoencoder for Zero Shot Learning 零样本学习 (论文、算法、数据集、代码)

DeepLearning | Relational Knowledge Transfer for Zero Shot Learning 零样本学习（论文、算法、数据集、代码）

一、AWA2数据集简介1.1 classes.txt1.2 JPGEImages1.3 licenses1.4 predicate-matrix-binary.txt1.5 predicate-matrix-continuous.txt1.6 predict-matrix.png1.7 predicate.txt1.8 README-attributes.txt和README-images.txt1.9 testclass.txt1.10 trainclasses.txt 二、数据集处理2.1 图片读取2.2 准备属性标签2.3 使用预训练的resnet101提取图片特征三、处理完毕的数据四、资源下载

一、AWA2数据集简介

该数据集是C. H. Lampert 等人在 Zero-Shot Learning - A Comprehensive Evaluation of the Good, the Bad and the Ugly上公布的动物识别数据集，该数据集一共包含以下几个文件

接下来我会一一对这些文件进行介绍

1.1 classes.txt

该文件记录了数据集所包含的动物种类，共50种，注意，该文件我稍微做了修改，源文件是没有+号的如6这里，这么做是为了写法保持一致。源数据集有的文件里用了加号，有的没用，这里统一了一下

1.2 JPGEImages

该文件夹包含了数据集的所有图片数据，格式如下，每一个子文件夹包含一种动物的图片

1.3 licenses

该文件夹包含每一张图片的授权，这个文件我们在处理时是用不到的

1.4 predicate-matrix-binary.txt

该文件记录了50种动物，每一种动物的85种属性特征情况，是一个50x85的矩阵，1表示有该特征，0表示无，如下

1.5 predicate-matrix-continuous.txt

和 predicate-matrix-binary.txt 文件一样，记录了50种动物，每一种动物的85种属性特征情况，只是该矩阵对属性的描述用的是连续数字

1.6 predict-matrix.png

文件 predicate-matrix-binary.txt 的图形化

1.7 predicate.txt

该文件记录了85种预测的属性分别是什么

1.8 README-attributes.txt和README-images.txt

这两个说明文件对我们也是没有用的

1.9 testclass.txt

该文件说明了哪些动物是测试种类，共10个测试类别

1.10 trainclasses.txt

该文件说明了哪些动物是训练种类，共40个训练类别

这里就介绍完了数据集的全部文件，简而言之，数据集包含50个种类动物的37322张图片，训练集40类30337张图片，测试集10类6985张图片

二、数据集处理

2.1 图片读取

这一步我们需要将图片统一大小为224x224x3，并为数据集制作相应的标签，代码如下

import pandas as pdimport osimport numpy as npimport cv2from PIL import Imageimage_size = 224# 指定图片大小path = '/Users/zhuxiaoxiansheng/Desktop/Animals_with_Attributes2/' #文件读取路径classname = pd.read_csv(path+'classes.txt',header=None,sep = '\t')dic_class2name = {classname.index[i]:classname.loc[i][1] for i in range(classname.shape[0])} dic_name2class = {classname.loc[i][1]:classname.index[i] for i in range(classname.shape[0])}# 两个字典，记录标签信息，分别是数字对应到文字，文字对应到数字#根据目录读取一类图像，read_num指定每一类读取多少图片，图片大小统一为image_sizedef load_Img(imgDir,read_num = 'max'):imgs = os.listdir(imgDir)imgs = np.ravel(pd.DataFrame(imgs).sort_values(by=0).values)if read_num == 'max':imgNum = len(imgs)else:imgNum = read_numdata = np.empty((imgNum,image_size,image_size,3),dtype="float32")print(imgNum)for i in range (imgNum):img = Image.open(imgDir+"/"+imgs[i])arr = np.asarray(img,dtype="float32")if arr.shape[1] > arr.shape[0]:arr = cv2.copyMakeBorder(arr,int((arr.shape[1]-arr.shape[0])/2),int((arr.shape[1]-arr.shape[0])/2),0,0,cv2.BORDER_CONSTANT,value=0)else:arr = cv2.copyMakeBorder(arr,0,0,int((arr.shape[0]-arr.shape[1])/2),int((arr.shape[0]-arr.shape[1])/2),cv2.BORDER_CONSTANT,value=0) #长宽不一致时，用padding使长宽一致arr = cv2.resize(arr,(image_size,image_size))if len(arr.shape) == 2:temp = np.empty((image_size,image_size,3))temp[:,:,0] = arrtemp[:,:,1] = arrtemp[:,:,2] = arrarr = temp data[i,:,:,:] = arrreturn data,imgNum #读取数据def load_data(train_classes,test_classes,num):read_num = numtraindata_list = []trainlabel_list = []testdata_list = []testlabel_list = [] for item in train_classes.iloc[:,0].values.tolist():tup = load_Img(path+'JPEGImages/'+item,read_num=read_num)traindata_list.append(tup[0])trainlabel_list += [dic_name2class[item]]*tup[1]for item in test_classes.iloc[:,0].values.tolist():tup = load_Img(path+'JPEGImages/'+item,read_num=read_num)testdata_list.append(tup[0])testlabel_list += [dic_name2class[item]]*tup[1]return np.row_stack(traindata_list),np.array(trainlabel_list),np.row_stack(testdata_list),np.array(testlabel_list)train_classes = pd.read_csv(path+'trainclasses.txt',header=None)test_classes = pd.read_csv(path+'testclasses.txt',header=None)traindata,trainlabel,testdata,testlabel = load_data(train_classes,test_classes,num='max')print(traindata.shape,trainlabel.shape,testdata.shape,testlabel.shape)#降图像和标签保存为numpy数组，下次可以直接读取np.save(path+'AWA2_224_traindata.npy',traindata)np.save(path+'AWA2_224_testdata.npy',testdata)np.save(path+'AWA2_trainlabel.npy',trainlabel)np.save(path+'AWA2_testlabel.npy',testlabel)

2.2 准备属性标签

刚刚我们读取了数据并制作了0-49的数字标签，但光是数字标签在零样本学习中是不足的，我们还需要每一张图片与其对应的属性标签

下面制作了连续属性的标签，同样的方法还可以制作离散（01）属性的标签，还可以将连续属性规范到0-1范围内作为标签，这些代码不再重复，处理好的标签会在最后的链接中统一给出

import pandas as pdimport numpy as nppath = '/Users/zhuxiaoxiansheng/Desktop/Animals_with_Attributes2/'def make_attribute_label(trainlabel,testlabel): attribut_bmatrix = pd.read_csv(path+'predicate-matrix-continuous.txt',header=None,sep = ',')trainlabel = pd.DataFrame(trainlabel).set_index(0)testlabel = pd.DataFrame(testlabel).set_index(0)return trainlabel.join(attribut_bmatrix),testlabel.join(attribut_bmatrix)trainlabel = np.load(path+'AWA2_trainlabel.npy')testlabel = np.load(path+'AWA2_testlabel.npy')train_attributelabel,test_attributelabel = make_attribute_label(trainlabel,testlabel)np.save(path+'AWA2_train_continuous_attributelabel.npy',train_attributelabel.values)np.save(path+'AWA2_test_continuous_attributelabel.npy',test_attributelabel.values)

2.3 使用预训练的resnet101提取图片特征

在零样本学习中，很多情况下，我们不会直接使用图片本身，使用卷积网络提取出的特征会更加方便

import numpy as npimport pandas as pdfrom sklearn.metrics import accuracy_scoreimport torchimport torchvisionfrom torchvision import datasets, models,transformsfrom torch.autograd import Variablefrom torch.utils.data import DataLoader,Datasetfrom tqdm import tqdmfrom torch import nn,optimimport lightgbm as lgbimport warningswarnings.filterwarnings("ignore")from sklearn.linear_model import LogisticRegressionpath = '/Users/zhuxiaoxiansheng/Desktop/Animals_with_Attributes2/'classname = pd.read_csv(path+'classes.txt',header=None,sep = '\t')dic_class2name = {classname.index[i]:classname.loc[i][1] for i in range(classname.shape[0])}dic_name2class = {classname.loc[i][1]:classname.index[i] for i in range(classname.shape[0])}def make_test_attributetable(): #制作测试10类的属性表attribut_bmatrix = pd.read_csv(path+'predicate-matrix-binary.txt',header=None,sep = ' ')test_classes = pd.read_csv(path+'testclasses.txt',header=None)test_classes_flag = []for item in test_classes.iloc[:,0].values.tolist():test_classes_flag.append(dic_name2class[item])return attribut_bmatrix.iloc[test_classes_flag,:]class dataset(Dataset):def __init__(self,data,label,transform):super().__init__()self.data = dataself.label = labelself.transform = transformdef __getitem__(self,index):return self.transform(self.data[index]),self.label[index]def __len__(self):return self.data.shape[0] class FeatureExtractor(nn.Module):def __init__(self, submodule, extracted_layers):super(FeatureExtractor,self).__init__()self.submodule = submoduleself.extracted_layers= extracted_layersdef forward(self, x):outputs = []for name, module in self.submodule._modules.items():if name is "fc": x = x.view(x.size(0), -1)x = module(x)if name in self.extracted_layers:outputs.append(x)return outputstraindata = np.load(path+'AWA2_224_traindata.npy')trainlabel = np.load(path+'AWA2_trainlabel.npy')train_attributelabel = np.load(path+'AWA2_train_attributelabel.npy')testdata = np.load(path+'AWA2_224_testdata.npy')testlabel = np.load(path+'AWA2_testlabel.npy')test_attributelabel = np.load(path+'AWA2_test_attributelabel.npy')print(traindata.shape,trainlabel.shape,train_attributelabel.shape)print(testdata.shape,testlabel.shape,test_attributelabel.shape)data_tf = pose([transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])])train_dataset = dataset(traindata,trainlabel,data_tf)test_dataset = dataset(testdata,testlabel,data_tf)train_loader = DataLoader(train_dataset,batch_size=1,shuffle=False)test_loader = DataLoader(test_dataset,batch_size=1,shuffle=False)model = models.resnet101(pretrained=True)#使用训练好的resnet101(if torch.cuda.is_available():model=model.cuda()model.eval()exact_list = ['avgpool'] #提取最后一层池化层的输出作为图像特征myexactor = FeatureExtractor(model,exact_list)train_feature_list = []for data in tqdm(train_loader):img,label = data if torch.cuda.is_available():with torch.no_grad():img = Variable(img).cuda()with torch.no_grad():label = Variable(label).cuda()else:with torch.no_grad():img = Variable(img)with torch.no_grad():label = Variable(label) feature = myexactor(img)[0]feature = feature.resize(feature.shape[0],feature.shape[1])train_feature_list.append(feature.detach().cpu().numpy()) trainfeatures = np.row_stack(train_feature_list) test_feature_list = []for data in tqdm(test_loader):img,label = data if torch.cuda.is_available():with torch.no_grad():img = Variable(img).cuda()with torch.no_grad():label = Variable(label).cuda()else:with torch.no_grad():img = Variable(img)with torch.no_grad():label = Variable(label) feature = myexactor(img)[0]feature = feature.resize(feature.shape[0],feature.shape[1])test_feature_list.append(feature.detach().cpu().numpy()) testfeatures = np.row_stack(test_feature_list) print(trainfeatures.shape,testfeatures.shape)