Deep Learning

design and train a classifier and a generative model using the CIFAR-100 dataset. While this might seem straightforward, the challenge is that you are only allowed to train small models (with a limited number of total parameters) for a short amount of training time (gradient update steps). You are also to write a short scientific report detailing both methods, experimental results, and limitations in a provided LATEX template that closely follows parts of the ICLR conference style guidelines. These files must be zipped together like this, replacing the username with your CIS username. You may not submit additional source code files: [login to view URL] [login to view URL] [login to view URL] (or .py) [login to view URL] (or .py) To assist in this, the following template reports and starter code are provided to build on: W [Deep Learning Paper Template] - login with Durham email, on ‘[login to view URL]’ click ‘make a copy’ to edit W [Google Colab Discriminative Model Starter Code] W [Google Colab Generative Model Starter Code] The deep discriminative model Design and train a CIFAR-100 classifier, reporting both the training and testing accuracy as in the above starter code. However, your model must have fewer than 100,000 (one hundred thousand) parameters (this is a low number of parameters for such a task). To display the total number of parameters for a deep neural network N, you can use something similar to the following code: print(f'> Number of parameters {len([login to view URL] to vector([login to view URL]()))}') Furthermore, you may not train the neural network for more than 10,000 (ten thousand) optimisation steps. You must clearly state the total number of parameters and optimisation steps in both the code and report. Exceeding either of these constraints will result in a 5 mark penalty for every 10% exceeded. For example, a model of 110,000 parameters incurs -5 marks whereas 150,000 parameters incurs -25 marks. A model trained for 11,000 steps incurs -5 marks and training for 15,000 steps incurs -25 marks. You should design the architecture yourself based on content covered in the lectures, practicals, and supplementary reading. If you reference existing code, this must be cited clearly in both the submitted python code and in the .pdf report. As in the guidance paper template, your report should include: (i) a plot of the training and test accuracy over the length of your training, (ii) the total number of parameters in your network, (iii) the final values for training loss, training accuracy and test accuracy (means and standard deviations) as in the format provided to you in the discriminative model starter code. You will be assessed primarily based on the quality of the report and the model accuracy. Further details of how this is graded are given in the marking scheme. The deep generative model Using the CIFAR-100 dataset, train a deep generative model to synthesise unique images that will be judged on their realism, diversity and uniqueness from the original training data. Limitations in terms of model size and training length also apply to the generative modelling task. Your model must have fewer than 1,000,000 (one million) parameters. Note that if your approach consists of multiple networks (e.g., like a GAN), the parameter limit applies to the whole model with all the networks combined (e.g., the parameter count of the generator and the discriminator added together should be less that 1,000,000). Furthermore, you may not train the neural network for more than 50,000 (fifty thousand) optimisation steps. Each optimisation step is counted when your entire model is trained by calculating the gradients for the whole model once. You must clearly state the total number of parameters and optimisation steps for the generative model of your choice in both the code and report. Exceeding either of these constraints will result in a 5 mark penalty for every 10% exceeded, for example, a model of 1,100,000 parameters incurs -5 marks whereas 1,500,000 parameters incurs -25 marks. A generative model trained for 55,000 steps incurs -5 marks and training for 75,000 steps incurs -25 marks. In the report, you must display (i) a unique batch of 64 non cherry-picked model samples, (ii) interpolations between 8 pairs of your samples, and (iii) you must provide FID scores between 10k model samples and the 10k images in the CIFAR-100 test dataset. You are not permitted to train on the test data. You can use CIFAR-100 class labels to aid your training and sampling without penalty. You are permitted to train on a subset of the CIFAR-100 dataset, but you will not score as highly in diversity by doing so. Your model should generate samples based on a noise vector z, drawn from a prior distribution, rather than being conditioned on x. In other words, the samples should not be derived from a function of x during inference.