微信客服：xiaoxionga100
微信客服：ITCS521
FIT 5124-无代写-Assignment 3
时间：2023-05-07
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
Assignment 3: Privacy Attacks and Defences in Machine
Learning Systems
Total marks 100
Task 3 and Task 4 Due on Week 11
(11:59:59am, 20 May, 2023, Firm!)
Task 5 and Task 6 Due on Week 14
(11:59:59am, 10 June, 2023, Firm!)
1 Overview
Machine learning models are widely used in various fields, but they can reveal
sensitive information about the individuals and data they are trained on. Revealing
sensitive information is a threat to privacy, especially when personal or sensitive data
is involved. This assignment explores two attacks on machine learning models:
Model Extraction and Membership Inference. The purpose of this assignment is to
understand these attacks, their methods, their impact on security and privacy, and
possible defence mechanisms.
You are expected to explain the objectives and threat models of these attacks based
on real-world examples, and design and implement them with countermeasures.
These tasks evaluate your understanding of security and privacy risks associated
with machine learning, and how to mitigate them.
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
X. Key steps and timeline
Attack Type Step Marks Time estimate Submit in…
Model Extraction
3.1, 3.2 Attack Description and design 6+6 5 hours
Week 11
4.1, 4.2 Defence Description and design 6+6 5 hours
3.3, 3.4 Implementation, Evaluation and
Analysis
6+6
8 hours
4.3, 4.4, 4.5 Implementation, Evaluation
and Analysis (supported by ChatGPT)
7+4+3
8 hours
Part 1 Total 12 + 12 + 12 + 14 = 50 26 hours
Member
Inference
5.1, 5.2 Attack Description and design 6+6 5 hours
Week 14
6.1, 6.2 Defence Description and Design 6+6 5 hours
5.3, 5.4 Implementation (ChatGPT
supported) and analysis
6+6
7 hours
6.3, 6.4 Implementation, Evaluation and
Analysis
7+4+3
8 hours
Part 2 Total 12 + 12 + 12 + 14 = 50 25 hours
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
2 Submission Policy
You need to submit a report (one single PDF file) to describe what you have done
and what you have observed with screenshots whenever necessary; you also need
to provide explanations or codes for the observations that are related to the tasks. In
your report, you are expected to answer all the questions listed in this manual.
Typeset your report into .pdf format (make sure it can be opened with Adobe
Reader) and name it as the format:
[Your Name]-[Student ID]-FIT5124-Assignment3,
HarryPotter-12345678-FIT5124-Assignment3-part1.pdf. for the first submission, and
HarryPotter-12345678-FIT5124-Assignment3-part2.pdf. for the second submission.
Please upload the PDF file to Moodle. Note that the first part if the assignment (Task
3 and Task 4) is due on 20 May 2023, Saturday, 11:59:59 am (No extension), and
the second part of the assignment (Task 3 and Task 4) is due on 10 June 2023,
Saturday, 11:59:59 am (No extension)
Late submission penalty: 10-point deduction per day. If you require special
consideration, the application should be submitted at least three days in
advance via Monash Connect (https://www.monash.edu/connect). Zero
tolerance on plagiarism: If you are found cheating, penalties will be applied, i.e., a
zero grade for the unit. The demonstration video is also used to detect/avoid
plagiarism. University policies can be found at
https://www.monash.edu/students/academic/
policies/academic-integrity
Generative AI tools are not restricted for this assessment task: In this
assessment, you can use generative artificial intelligence (AI) to assist you in any
way. Any use of generative AI must be appropriately acknowledged. How to cite the
use of generative AI can be found here:
https://www.monash.edu/learning-teaching/teachhq/Teaching-practices/artificial-intelli
gence/policy-and-practice-guidance-around-acceptable-and-responsible-use-of-ai-te
chnologies. Note that there will be two subtasks that require you to use ChatGPT to
complete (Task 4.5 and Task 5.3).
3 Stealing Machine Learning Model (25 marks)
Model extraction attacks involve creating a new machine learning model (MLM) that
mimics the behaviour of an existing MLM. These attacks can be used to steal
valuable MLMs for competitive advantage or for malicious activities like analysing the
model for adversarial attacks. Your task is to:
● Implement a model extraction attack using PyTorch
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
○ Include documentation and comments to support your approach.
● Evaluate the performance of the extracted model and compare it to the target
model's performance.
Some references (you may follow other references/realisations/algorithms as long as
they can achieve the same functionality):
● Dive into Deep Learning
https://d2l.ai/chapter_multilayer-perceptrons/index.html: a book that helps you
understand the details of MLP and neural networks.
● PyTorch tutorials: https://pytorch.org/tutorials/
● MNIST dataset website: http://yann.lecun.com/exdb/mnist/
To complete this assignment, you need to train a neural network model that can
accurately classify handwritten digits in the MNIST dataset as your target model. The
MNIST dataset is available in various formats, such as CSV, MATLAB, and NumPy
arrays. You can use the PyTorch DataLoader to load the dataset as shown below:
train_set = torchvision.datasets. MNIST(
root = './data/MNIST',
train = True,
download = True,
transform = transforms.Compose([transforms.ToTensor()])
)
test_set = torchvision.datasets. MNIST(
root = './data/MNIST',
train = False,
transform = transforms.Compose([transforms.ToTensor()])
)
To implement your target model, you can use PyTorch. We have provided a sample
code for model training in the file "a3_mnist.py". You can read and execute the code
to train the model and evaluate its performance. The trained model will be saved in
the format "target_model.pth". To save and load the PyTorch model, you can use the
following code:
# Save the model
torch.save(model.state_dict(), 'target_model.pth')
# Load the model
model = My_MNIST()
model.load_state_dict(torch.load('target_model.pth'))
model.eval()
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
Note that "target_model.pth" will only store the model parameters, not the model
structure. Therefore, when you load the saved parameters, you need to always
redefine the same model and load the parameters to the model. In this assignment,
you should always add "from as_mnist import My_MNIST" at the beginning of your
code implementation to import the model structure if you want to load this model.
3.1 Attack Description (6 Marks)
To design a practical model extraction attack, you should first specify the attack
settings. In this subtask, your goal is to define your attack objective, provide a
detailed threat model, and explain why the attackers you considered are practical,
and explain why your settings are practical.
● Attack Objective: describe the specific objective of your attack, such as the
information you are trying to extract from the target model (e.g., model
parameters, architectures, a mimic model with similar prediction/accuracy).
(2 mark)
● Attacker's Capability: describe the attacker’s capabilities, including any
assumptions about the attacker's
o access to the target model training process,
o target trained model,
o data used to train the model,
o any other information.
(2 marks)
● Practical Scenario Example: provide a detailed explanation of your model
extraction attack. Explain your
o attack objective (a real-world scenario and potential impact).
o attacker’s capabilities (a real-world scenario where your attacker has
these capabilities).
(2 marks)
3.2 Attack Design (6 Marks)
In this subtask, explain the process of your designed model extraction attack with
a step-by-step description. Justify your attack design by explaining:
● why you included these steps,
● how they contribute to your attack goals,
● why they are suitable for the threat model you described in the previous subtask.
You can draw a figure to support the high-level description of your attacks.
3.3 Implementation and Evaluation (6 Marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
After designing the attack, the next step is implementation. In this subtask, you will
implement and evaluate the model extraction attack described in Subtask 3.3.
Please include:
● Code implementation: provide your code implementation with detailed
explanations and comments. Including a screenshot and detailed logs will
support and communicate this. (3 marks)
● Results and evaluation: Explain how you evaluated the attack’s performance
and report the results. Justify the chosen evaluation metrics based on the
attack goals described in Subtask 3.2. (3 marks)
3.4 Attack Analysis (7 Marks)
In this subtask, you will evaluate the effectiveness and efficiency of your model
extraction attack and analyse the factors that impact its success. You should include:
● Effectiveness: Based on your results from task 3.4, evaluate whether your
attack has met the goal you set in task 3.2. (1 mark)
● Efficiency: Measure the time it takes to extract the target model information
and discuss any other resource requirements, such as data size or cost.
Compare the cost of your attack to the training cost for you to develop the
target model, and determine whether your attack is practical. (2 mark)
● Impact Factors: Conduct experiments to investigate the impact of different
o hyperparameters,
o data amount and type,
o attacker access to the target model and training data.
(2 marks)
● Dangers and Limitations: discuss the potential dangers and limitations of your
proposed model extraction attack based on the practical scenario you
considered in Task 3.1. (2 mark)
4 Defending against model stealing attack (25 marks)
Task 3 showed us that model extraction attacks could pose a serious threat to the
security of machine learning models. This makes it critical to develop effective
defences against such attacks. In this task, you will design and implement a defence
against the attack you designed.
For this task, you will
1. Research existing defences against model extraction attacks.
2. Select the most suitable approach for your specific attack setting.
3. Create and implement your own defence.
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
4. Evaluate the performance of your defence under a variety of experimental
settings.
5. Analyse the results to identify any design limitations.
4.1 Defence Overview (6 Marks)
Similar to developing your model extraction attacks, we will now specify the settings
for your defence. Your objective in this subtask is to provide an overview of your
defence by defining your defence goal, providing a detailed setting for your defence,
and explaining why your settings are practical. Specifically, include the following:
● Defence Objective: describe the specific objective of your defence, such as
o what information you aim to protect from the model extraction attack
o how you measure the success of your defence (e.g. reduction in
extracted performance, or the increase in attack cost).
(2 mark)
● Defender’s Capability: describe the capabilities of the defender, including any
assumptions you make about
o their access to the target model training/inference process,
o target trained model,
o data used to train the model,
o any other information.
(2 marks)
● Practical Scenario Example: provide a detailed explanation of your defence.
Explain your
o defence objective (a real-world scenario where your defence can be
applied),
o defender’s capability (a real-world scenario where your defender has
these capabilities),
(2 marks)
Note: Your defence settings should be compatible with the attacks you defined in the
previous task. Explain how your defence settings (goals/capabilities/practical
example) are suitable when mitigating your attacks.
4.2 Defence Design (6 Marks)
In this subtask, you will explain the steps involved in your designed defence
mechanism and justify your design choices, including
● a step-by-step description of the defence mechanism;
● figures or diagrams to help provide a high-level overview of your defence.
Justify your defence design by explaining why
● each step is included,
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
● their contribution to your defence goals,
● how they address the attack developed in Task 3.
4.3 Implementation and Evaluation (6 Marks)
In this subtask, you will implement and evaluate your defence against the model
extraction attack you described in Task 3. Include the following elements:
● Code Implementation: provide a screenshot of your code implementation
and explain it in detail. Include detailed comments to your code and print
detailed logs to help others understand them. (3 marks)
● Results and Evaluation: specify how you evaluated the performance of your
defence and report the results. You should explain why you chose specific
evaluation metrics based on the defence goals outlined in Task 4.1. (3 marks)
4.4 Defence Analysis (4 Marks)
In this subtask, you will analyse the effectiveness and efficiency of your defence
against the model extraction attack. Specifically, you should include:
● Effectiveness: Explain whether your defence meets the objectives set in Task
4.1 based on the results obtained in Task 4.3. (1 mark)
● Efficiency: Evaluate and discuss the costs associated with deploying your
defence mechanism, including time, computational complexity, or other
required resources. Discuss whether your defence can be used in practice. (1
mark)
● Impact factors: Conduct experiments to investigate any factors that affect the
performance of your defence. For example, you could examine the impact of
different hyperparameters, the amount and type of required data, or the
defender's capabilities. (2 marks)
4.5 Using ChatGPT for Further Analysis (3 Marks)
In this subtask, you will explore the effectiveness of your defence methods against
an advanced attacker who already knows your defence strategies. In addition, you
will use ChatGPT to help answer this question! Specifically, to accomplish this task,
please follow the steps below:
● Explain your defence method to ChatGPT, as you have done in Task 3.4. (1
mark)
● Define the advanced attacker to ChatGPT, including their capabilities and
knowledge of your defence strategy. (1 mark)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
(Hints: You may define it by considering additional attacker’s knowledge
corresponding to your defence, such as one or several steps/processes of
your defence, or any relevant hyperparameters.)
● Ask ChatGPT whether your defence still works against the advanced attacker
you defined above.
● Identify whether the answer from ChatGPT is correct. If the answer is
incorrect, please provide your answer. If there are only minor errors in the
response, correct them. Be sure to include your questions and the
original answers from ChatGPT in your report. (1 mark)
Note: To get a better answer from ChatGPT, you should provide as much detailed
information as possible when explaining your defence and defining the advanced
attacker. Please also include the prompts and responses here for marking purposes.
5 Membership Inference (25 marks)
In this assignment, you will implement a membership inference attack on the target
model from Task 3.1. The goal of a membership inference attack is to determine if
a particular sample was part of the training data used to create the model. These
attacks can uncover sensitive information about individuals whose data was used to
train the model. For example, healthcare data is highly sensitive, and membership
inference attacks on this data can reveal details about an individual's health status,
diagnoses, and treatment history.
Your task is to implement a membership inference attack by completing each
subtask and evaluating its performance. Finally, you will provide a detailed analysis
of your results.
5.1 Attack Overview (5 Marks)
To design a practical membership inference attack, we will first specify the attack
settings. The goal is to determine whether a sample was included in the model
training data. In this subtask, your objective is to provide an overview of your attack
by defining a detailed threat model and explaining why your settings are practical.
You should include the following items:
● Attacker's Capability: describe the attacker’s capabilities assumptions
regarding their access to the target model training/inference process and
trained model. (2 marks)
● Practical Scenario Example: provide a detailed explanation of your
membership inference attack, including a real-world scenario in which the
attack could be used and the potential impact of the attack). Explain the
attacker’s capability in a real-world scenario. (3 marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
5.2 Attack Design (6 Marks)
In this subtask, explain the process of your designed membership inference attack
including a step-by-step description of the attacks. You can draw a figure to describe
the high-level overview of your attacks. You should also justify your attack design by
explaining why you include these steps in your attacks and how they can contribute
to your attack goals and are suitable for the threat model you described in Task 5.1.
5.3 Using ChatGPT for Implementation (8 Marks)
In Task 4, we used ChatGPT for defence analysis. ChatGPT can also provide
guidance and code snippets to help you in the implementation process.
In this subtask, you will implement a membership inference attack using PyTorch.
We will still use ChatGPT to help us with the coding by following the steps below:
● Ask ChatGPT to provide a PyTorch code for a simple membership inference
attack against a neural network model training on the MNIST dataset. Include
your prompts and the original answers from ChatGPT in your report. (1
mark)
● Identify whether the attack implemented by ChatGPT has the same goal and
capabilities as defined in Task 5.2. If not, provide a screenshot demonstrating
your revised code implementation and provide a detailed explanation.
Include detailed comments to your code and print detailed logs to help others
understand them. (4 marks)
● Results and evaluation: Specify how you evaluate the performance of your
attack and include the evaluation in your code. You can refine your code to
report the results. Additionally, you should explain why you chose the specific
evaluation metrics based on your attack goals in Task 5.2. (3 marks)
5.4 Attack Analysis (6 Marks)
In this subtask, you will analyse how effective and efficient your membership
inference attack is, and explore the factors that influence its success rate. Your
analysis should include:
● Effectiveness and efficiency: As in Task 3.5, evaluate the effectiveness and
cost of your attack, and discuss whether it is practical. (1 mark)
● Impact factors of the attacker’s capability: Conduct experiments to investigate
how an attacker’s capability affects your attack. For example, you could test
different hyperparameters, data requirements, and access to the target model
and training data. (2 marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
● Impact factors of the target model: conduct experiments to investigate how
different target models are vulnerable to membership inference attacks. For
example, you could investigate the impact of target model training settings like
training epochs and model architecture. (2 marks)
● Danger and limitations: discuss the potential dangers and limitations of your
proposed membership inference attacks based on the practical scenario you
considered in Task 5.1. (1 mark)
6 Defending against membership inference attack (25 marks)
In Task 5, we saw that membership inference attacks could seriously threaten the
privacy and security of training data. Your task now is to create a defence against the
attack you designed. This will help protect the privacy of the training data and
prevent attackers from identifying whether a particular data point was used in the
training set.
To do this, you can research existing defences against membership inference
attacks and select the most appropriate ones for your specific attack setting
described in the previous task. You will then develop and implement your own
defence. You will also evaluate the performance of your defence under various
experimental settings and analyse the results to identify any limitations of your
defence.
6.1 Defense Overview (6 Marks)
In this next subtask, you will define the settings for your defence against membership
inference attacks. Like with developing your attacks, you should specify your
defence settings by outlining your defence goal, providing details about the defence
setting, and explaining why your settings are practical. Your defence settings should
be compatible with the attacks you designed in the previous task. Specifically, you
should include:
● Defender’s Capability: Describe the capabilities of the defender, including
any assumptions you are making about its access to the target model
training/inference process, target trained model, the data used to train the
model, or other information. (3 marks)
● Practical Scenario Example: Provide a detailed explanation of your defence,
including its objective (a real-world scenario in which your defence could be
applied) and your defender’s capabilities (a real-world scenario in which your
defender can have these capabilities). (3 marks)
Note: You should also explain how your defence settings (goals/capabilities/practical
example) make it suitable when mitigating your attacks. Finally, you will need to
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
evaluate the effectiveness of your defence and provide an analysis of its limitations
under different experimental conditions."
6.2 Defence Design (6 Marks)
In this subtask, you will explain how your defence mechanism works. You should
describe the steps involved in your defence mechanism and include any diagrams or
figures to help explain it. You should also explain why you included these steps and
how they contribute to your defence goals, as well as how they are suitable for the
attack described in Task 5.
6.3 Implementation and Evaluation (6 Marks)
In this subtask, you will implement and evaluate your defence against the
membership inference attack described in Task 5. Include the following elements:
● Code implementation: provide a screenshot of your code implementation
along with detailed explanations and comments. (3 marks)
● Results and evaluation: Report the results of your evaluation, and explain the
specific evaluation metrics you chose based on your defence goals in Task
6.1. (3 marks)
6.4 Defence Analysis (7 Marks)
In this subtask, analyse the effectiveness and efficiency of your defence against
membership inference attacks. You should include:
● Defence cost: Based on your results in Task 6.3, discuss the deployment cost
of your defence method (including time cost, computation complexity, or any
other required resources). Discuss whether your defence can be used in
practice. (1 mark)
● Impact factors: Conduct experiments to investigate the factors that affect the
performance of your defence. For example, you could investigate the impact
of different hyperparameters, the amount and type of required data, or the
defender's capabilities. (2 marks)
● Against advanced attacker: Discuss whether your defence still works against
the advanced attacker. (4 marks)
o If your defence works against the advanced attacker, please provide an
explanation of what the advanced attacker knows about your defence
and why your defence still remains effective.
o If you think your defence is not effective against the advanced attacker,
explain why, and provide an explanation of the maximum capacity of
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
your defence, which refers to the strongest attacker your design can
defend against.
(Hints: You may follow the same steps as Task 4.5 but without using
ChatGPT.)
Assignment 3: Privacy Attacks and Defences in Machine
Learning Systems
Total marks 100
Task 3 and Task 4 Due on Week 11
(11:59:59am, 20 May, 2023, Firm!)
Task 5 and Task 6 Due on Week 14
(11:59:59am, 10 June, 2023, Firm!)
1 Overview
Machine learning models are widely used in various fields, but they can reveal
sensitive information about the individuals and data they are trained on. Revealing
sensitive information is a threat to privacy, especially when personal or sensitive data
is involved. This assignment explores two attacks on machine learning models:
Model Extraction and Membership Inference. The purpose of this assignment is to
understand these attacks, their methods, their impact on security and privacy, and
possible defence mechanisms.
You are expected to explain the objectives and threat models of these attacks based
on real-world examples, and design and implement them with countermeasures.
These tasks evaluate your understanding of security and privacy risks associated
with machine learning, and how to mitigate them.
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
X. Key steps and timeline
Attack Type Step Marks Time estimate Submit in…
Model Extraction
3.1, 3.2 Attack Description and design 6+6 5 hours
Week 11
4.1, 4.2 Defence Description and design 6+6 5 hours
3.3, 3.4 Implementation, Evaluation and
Analysis
6+6
8 hours
4.3, 4.4, 4.5 Implementation, Evaluation
and Analysis (supported by ChatGPT)
7+4+3
8 hours
Part 1 Total 12 + 12 + 12 + 14 = 50 26 hours
Member
Inference
5.1, 5.2 Attack Description and design 6+6 5 hours
Week 14
6.1, 6.2 Defence Description and Design 6+6 5 hours
5.3, 5.4 Implementation (ChatGPT
supported) and analysis
6+6
7 hours
6.3, 6.4 Implementation, Evaluation and
Analysis
7+4+3
8 hours
Part 2 Total 12 + 12 + 12 + 14 = 50 25 hours
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
2 Submission Policy
You need to submit a report (one single PDF file) to describe what you have done
and what you have observed with screenshots whenever necessary; you also need
to provide explanations or codes for the observations that are related to the tasks. In
your report, you are expected to answer all the questions listed in this manual.
Typeset your report into .pdf format (make sure it can be opened with Adobe
Reader) and name it as the format:
[Your Name]-[Student ID]-FIT5124-Assignment3,
HarryPotter-12345678-FIT5124-Assignment3-part1.pdf. for the first submission, and
HarryPotter-12345678-FIT5124-Assignment3-part2.pdf. for the second submission.
Please upload the PDF file to Moodle. Note that the first part if the assignment (Task
3 and Task 4) is due on 20 May 2023, Saturday, 11:59:59 am (No extension), and
the second part of the assignment (Task 3 and Task 4) is due on 10 June 2023,
Saturday, 11:59:59 am (No extension)
Late submission penalty: 10-point deduction per day. If you require special
consideration, the application should be submitted at least three days in
advance via Monash Connect (https://www.monash.edu/connect). Zero
tolerance on plagiarism: If you are found cheating, penalties will be applied, i.e., a
zero grade for the unit. The demonstration video is also used to detect/avoid
plagiarism. University policies can be found at
https://www.monash.edu/students/academic/
policies/academic-integrity
Generative AI tools are not restricted for this assessment task: In this
assessment, you can use generative artificial intelligence (AI) to assist you in any
way. Any use of generative AI must be appropriately acknowledged. How to cite the
use of generative AI can be found here:
https://www.monash.edu/learning-teaching/teachhq/Teaching-practices/artificial-intelli
gence/policy-and-practice-guidance-around-acceptable-and-responsible-use-of-ai-te
chnologies. Note that there will be two subtasks that require you to use ChatGPT to
complete (Task 4.5 and Task 5.3).
3 Stealing Machine Learning Model (25 marks)
Model extraction attacks involve creating a new machine learning model (MLM) that
mimics the behaviour of an existing MLM. These attacks can be used to steal
valuable MLMs for competitive advantage or for malicious activities like analysing the
model for adversarial attacks. Your task is to:
● Implement a model extraction attack using PyTorch
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
○ Include documentation and comments to support your approach.
● Evaluate the performance of the extracted model and compare it to the target
model's performance.
Some references (you may follow other references/realisations/algorithms as long as
they can achieve the same functionality):
● Dive into Deep Learning
https://d2l.ai/chapter_multilayer-perceptrons/index.html: a book that helps you
understand the details of MLP and neural networks.
● PyTorch tutorials: https://pytorch.org/tutorials/
● MNIST dataset website: http://yann.lecun.com/exdb/mnist/
To complete this assignment, you need to train a neural network model that can
accurately classify handwritten digits in the MNIST dataset as your target model. The
MNIST dataset is available in various formats, such as CSV, MATLAB, and NumPy
arrays. You can use the PyTorch DataLoader to load the dataset as shown below:
train_set = torchvision.datasets. MNIST(
root = './data/MNIST',
train = True,
download = True,
transform = transforms.Compose([transforms.ToTensor()])
)
test_set = torchvision.datasets. MNIST(
root = './data/MNIST',
train = False,
transform = transforms.Compose([transforms.ToTensor()])
)
To implement your target model, you can use PyTorch. We have provided a sample
code for model training in the file "a3_mnist.py". You can read and execute the code
to train the model and evaluate its performance. The trained model will be saved in
the format "target_model.pth". To save and load the PyTorch model, you can use the
following code:
# Save the model
torch.save(model.state_dict(), 'target_model.pth')
# Load the model
model = My_MNIST()
model.load_state_dict(torch.load('target_model.pth'))
model.eval()
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
Note that "target_model.pth" will only store the model parameters, not the model
structure. Therefore, when you load the saved parameters, you need to always
redefine the same model and load the parameters to the model. In this assignment,
you should always add "from as_mnist import My_MNIST" at the beginning of your
code implementation to import the model structure if you want to load this model.
3.1 Attack Description (6 Marks)
To design a practical model extraction attack, you should first specify the attack
settings. In this subtask, your goal is to define your attack objective, provide a
detailed threat model, and explain why the attackers you considered are practical,
and explain why your settings are practical.
● Attack Objective: describe the specific objective of your attack, such as the
information you are trying to extract from the target model (e.g., model
parameters, architectures, a mimic model with similar prediction/accuracy).
(2 mark)
● Attacker's Capability: describe the attacker’s capabilities, including any
assumptions about the attacker's
o access to the target model training process,
o target trained model,
o data used to train the model,
o any other information.
(2 marks)
● Practical Scenario Example: provide a detailed explanation of your model
extraction attack. Explain your
o attack objective (a real-world scenario and potential impact).
o attacker’s capabilities (a real-world scenario where your attacker has
these capabilities).
(2 marks)
3.2 Attack Design (6 Marks)
In this subtask, explain the process of your designed model extraction attack with
a step-by-step description. Justify your attack design by explaining:
● why you included these steps,
● how they contribute to your attack goals,
● why they are suitable for the threat model you described in the previous subtask.
You can draw a figure to support the high-level description of your attacks.
3.3 Implementation and Evaluation (6 Marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
After designing the attack, the next step is implementation. In this subtask, you will
implement and evaluate the model extraction attack described in Subtask 3.3.
Please include:
● Code implementation: provide your code implementation with detailed
explanations and comments. Including a screenshot and detailed logs will
support and communicate this. (3 marks)
● Results and evaluation: Explain how you evaluated the attack’s performance
and report the results. Justify the chosen evaluation metrics based on the
attack goals described in Subtask 3.2. (3 marks)
3.4 Attack Analysis (7 Marks)
In this subtask, you will evaluate the effectiveness and efficiency of your model
extraction attack and analyse the factors that impact its success. You should include:
● Effectiveness: Based on your results from task 3.4, evaluate whether your
attack has met the goal you set in task 3.2. (1 mark)
● Efficiency: Measure the time it takes to extract the target model information
and discuss any other resource requirements, such as data size or cost.
Compare the cost of your attack to the training cost for you to develop the
target model, and determine whether your attack is practical. (2 mark)
● Impact Factors: Conduct experiments to investigate the impact of different
o hyperparameters,
o data amount and type,
o attacker access to the target model and training data.
(2 marks)
● Dangers and Limitations: discuss the potential dangers and limitations of your
proposed model extraction attack based on the practical scenario you
considered in Task 3.1. (2 mark)
4 Defending against model stealing attack (25 marks)
Task 3 showed us that model extraction attacks could pose a serious threat to the
security of machine learning models. This makes it critical to develop effective
defences against such attacks. In this task, you will design and implement a defence
against the attack you designed.
For this task, you will
1. Research existing defences against model extraction attacks.
2. Select the most suitable approach for your specific attack setting.
3. Create and implement your own defence.
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
4. Evaluate the performance of your defence under a variety of experimental
settings.
5. Analyse the results to identify any design limitations.
4.1 Defence Overview (6 Marks)
Similar to developing your model extraction attacks, we will now specify the settings
for your defence. Your objective in this subtask is to provide an overview of your
defence by defining your defence goal, providing a detailed setting for your defence,
and explaining why your settings are practical. Specifically, include the following:
● Defence Objective: describe the specific objective of your defence, such as
o what information you aim to protect from the model extraction attack
o how you measure the success of your defence (e.g. reduction in
extracted performance, or the increase in attack cost).
(2 mark)
● Defender’s Capability: describe the capabilities of the defender, including any
assumptions you make about
o their access to the target model training/inference process,
o target trained model,
o data used to train the model,
o any other information.
(2 marks)
● Practical Scenario Example: provide a detailed explanation of your defence.
Explain your
o defence objective (a real-world scenario where your defence can be
applied),
o defender’s capability (a real-world scenario where your defender has
these capabilities),
(2 marks)
Note: Your defence settings should be compatible with the attacks you defined in the
previous task. Explain how your defence settings (goals/capabilities/practical
example) are suitable when mitigating your attacks.
4.2 Defence Design (6 Marks)
In this subtask, you will explain the steps involved in your designed defence
mechanism and justify your design choices, including
● a step-by-step description of the defence mechanism;
● figures or diagrams to help provide a high-level overview of your defence.
Justify your defence design by explaining why
● each step is included,
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
● their contribution to your defence goals,
● how they address the attack developed in Task 3.
4.3 Implementation and Evaluation (6 Marks)
In this subtask, you will implement and evaluate your defence against the model
extraction attack you described in Task 3. Include the following elements:
● Code Implementation: provide a screenshot of your code implementation
and explain it in detail. Include detailed comments to your code and print
detailed logs to help others understand them. (3 marks)
● Results and Evaluation: specify how you evaluated the performance of your
defence and report the results. You should explain why you chose specific
evaluation metrics based on the defence goals outlined in Task 4.1. (3 marks)
4.4 Defence Analysis (4 Marks)
In this subtask, you will analyse the effectiveness and efficiency of your defence
against the model extraction attack. Specifically, you should include:
● Effectiveness: Explain whether your defence meets the objectives set in Task
4.1 based on the results obtained in Task 4.3. (1 mark)
● Efficiency: Evaluate and discuss the costs associated with deploying your
defence mechanism, including time, computational complexity, or other
required resources. Discuss whether your defence can be used in practice. (1
mark)
● Impact factors: Conduct experiments to investigate any factors that affect the
performance of your defence. For example, you could examine the impact of
different hyperparameters, the amount and type of required data, or the
defender's capabilities. (2 marks)
4.5 Using ChatGPT for Further Analysis (3 Marks)
In this subtask, you will explore the effectiveness of your defence methods against
an advanced attacker who already knows your defence strategies. In addition, you
will use ChatGPT to help answer this question! Specifically, to accomplish this task,
please follow the steps below:
● Explain your defence method to ChatGPT, as you have done in Task 3.4. (1
mark)
● Define the advanced attacker to ChatGPT, including their capabilities and
knowledge of your defence strategy. (1 mark)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
(Hints: You may define it by considering additional attacker’s knowledge
corresponding to your defence, such as one or several steps/processes of
your defence, or any relevant hyperparameters.)
● Ask ChatGPT whether your defence still works against the advanced attacker
you defined above.
● Identify whether the answer from ChatGPT is correct. If the answer is
incorrect, please provide your answer. If there are only minor errors in the
response, correct them. Be sure to include your questions and the
original answers from ChatGPT in your report. (1 mark)
Note: To get a better answer from ChatGPT, you should provide as much detailed
information as possible when explaining your defence and defining the advanced
attacker. Please also include the prompts and responses here for marking purposes.
5 Membership Inference (25 marks)
In this assignment, you will implement a membership inference attack on the target
model from Task 3.1. The goal of a membership inference attack is to determine if
a particular sample was part of the training data used to create the model. These
attacks can uncover sensitive information about individuals whose data was used to
train the model. For example, healthcare data is highly sensitive, and membership
inference attacks on this data can reveal details about an individual's health status,
diagnoses, and treatment history.
Your task is to implement a membership inference attack by completing each
subtask and evaluating its performance. Finally, you will provide a detailed analysis
of your results.
5.1 Attack Overview (5 Marks)
To design a practical membership inference attack, we will first specify the attack
settings. The goal is to determine whether a sample was included in the model
training data. In this subtask, your objective is to provide an overview of your attack
by defining a detailed threat model and explaining why your settings are practical.
You should include the following items:
● Attacker's Capability: describe the attacker’s capabilities assumptions
regarding their access to the target model training/inference process and
trained model. (2 marks)
● Practical Scenario Example: provide a detailed explanation of your
membership inference attack, including a real-world scenario in which the
attack could be used and the potential impact of the attack). Explain the
attacker’s capability in a real-world scenario. (3 marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
5.2 Attack Design (6 Marks)
In this subtask, explain the process of your designed membership inference attack
including a step-by-step description of the attacks. You can draw a figure to describe
the high-level overview of your attacks. You should also justify your attack design by
explaining why you include these steps in your attacks and how they can contribute
to your attack goals and are suitable for the threat model you described in Task 5.1.
5.3 Using ChatGPT for Implementation (8 Marks)
In Task 4, we used ChatGPT for defence analysis. ChatGPT can also provide
guidance and code snippets to help you in the implementation process.
In this subtask, you will implement a membership inference attack using PyTorch.
We will still use ChatGPT to help us with the coding by following the steps below:
● Ask ChatGPT to provide a PyTorch code for a simple membership inference
attack against a neural network model training on the MNIST dataset. Include
your prompts and the original answers from ChatGPT in your report. (1
mark)
● Identify whether the attack implemented by ChatGPT has the same goal and
capabilities as defined in Task 5.2. If not, provide a screenshot demonstrating
your revised code implementation and provide a detailed explanation.
Include detailed comments to your code and print detailed logs to help others
understand them. (4 marks)
● Results and evaluation: Specify how you evaluate the performance of your
attack and include the evaluation in your code. You can refine your code to
report the results. Additionally, you should explain why you chose the specific
evaluation metrics based on your attack goals in Task 5.2. (3 marks)
5.4 Attack Analysis (6 Marks)
In this subtask, you will analyse how effective and efficient your membership
inference attack is, and explore the factors that influence its success rate. Your
analysis should include:
● Effectiveness and efficiency: As in Task 3.5, evaluate the effectiveness and
cost of your attack, and discuss whether it is practical. (1 mark)
● Impact factors of the attacker’s capability: Conduct experiments to investigate
how an attacker’s capability affects your attack. For example, you could test
different hyperparameters, data requirements, and access to the target model
and training data. (2 marks)
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
● Impact factors of the target model: conduct experiments to investigate how
different target models are vulnerable to membership inference attacks. For
example, you could investigate the impact of target model training settings like
training epochs and model architecture. (2 marks)
● Danger and limitations: discuss the potential dangers and limitations of your
proposed membership inference attacks based on the practical scenario you
considered in Task 5.1. (1 mark)
6 Defending against membership inference attack (25 marks)
In Task 5, we saw that membership inference attacks could seriously threaten the
privacy and security of training data. Your task now is to create a defence against the
attack you designed. This will help protect the privacy of the training data and
prevent attackers from identifying whether a particular data point was used in the
training set.
To do this, you can research existing defences against membership inference
attacks and select the most appropriate ones for your specific attack setting
described in the previous task. You will then develop and implement your own
defence. You will also evaluate the performance of your defence under various
experimental settings and analyse the results to identify any limitations of your
defence.
6.1 Defense Overview (6 Marks)
In this next subtask, you will define the settings for your defence against membership
inference attacks. Like with developing your attacks, you should specify your
defence settings by outlining your defence goal, providing details about the defence
setting, and explaining why your settings are practical. Your defence settings should
be compatible with the attacks you designed in the previous task. Specifically, you
should include:
● Defender’s Capability: Describe the capabilities of the defender, including
any assumptions you are making about its access to the target model
training/inference process, target trained model, the data used to train the
model, or other information. (3 marks)
● Practical Scenario Example: Provide a detailed explanation of your defence,
including its objective (a real-world scenario in which your defence could be
applied) and your defender’s capabilities (a real-world scenario in which your
defender can have these capabilities). (3 marks)
Note: You should also explain how your defence settings (goals/capabilities/practical
example) make it suitable when mitigating your attacks. Finally, you will need to
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
evaluate the effectiveness of your defence and provide an analysis of its limitations
under different experimental conditions."
6.2 Defence Design (6 Marks)
In this subtask, you will explain how your defence mechanism works. You should
describe the steps involved in your defence mechanism and include any diagrams or
figures to help explain it. You should also explain why you included these steps and
how they contribute to your defence goals, as well as how they are suitable for the
attack described in Task 5.
6.3 Implementation and Evaluation (6 Marks)
In this subtask, you will implement and evaluate your defence against the
membership inference attack described in Task 5. Include the following elements:
● Code implementation: provide a screenshot of your code implementation
along with detailed explanations and comments. (3 marks)
● Results and evaluation: Report the results of your evaluation, and explain the
specific evaluation metrics you chose based on your defence goals in Task
6.1. (3 marks)
6.4 Defence Analysis (7 Marks)
In this subtask, analyse the effectiveness and efficiency of your defence against
membership inference attacks. You should include:
● Defence cost: Based on your results in Task 6.3, discuss the deployment cost
of your defence method (including time cost, computation complexity, or any
other required resources). Discuss whether your defence can be used in
practice. (1 mark)
● Impact factors: Conduct experiments to investigate the factors that affect the
performance of your defence. For example, you could investigate the impact
of different hyperparameters, the amount and type of required data, or the
defender's capabilities. (2 marks)
● Against advanced attacker: Discuss whether your defence still works against
the advanced attacker. (4 marks)
o If your defence works against the advanced attacker, please provide an
explanation of what the advanced attacker knows about your defence
and why your defence still remains effective.
o If you think your defence is not effective against the advanced attacker,
explain why, and provide an explanation of the maximum capacity of
FIT 5124 Emerging Topics for Cybersecurity in Practice (S1 2023)
your defence, which refers to the strongest attacker your design can
defend against.
(Hints: You may follow the same steps as Task 4.5 but without using
ChatGPT.)
