GraphGen's built-in anti-overfitting strategy:
- Diversity safeguard mechanisms::
1. AdoptionstyleParametric control (concise/detailed/medical, etc.) generates expression variation
2. Multi-skip sampling automatically generates questions and answers from multiple perspectives on the same knowledge point.
3. Built-in Q&A reconstruction module generates different representations of the same semantics - Data validation program::
- existconfigs/graphgen_config.yamlenablediversity_check: true
- The output directory generatesdiversity_report.jsonIncludes repeat rate indicator
- It is recommended to maintain an entity repetition rate of <151 TP3T, which can be adjusted by increasing the input data volume - Training recommendations::
- A 1:2 mix of synthetic and real data is recommended.
- Priority is given to base models with parametric quantities of 7B and above.
- Monitor validation set loss early stopping (early stopping)
Project tests show that this scheme reduces the risk of overfitting by 671 TP3T (comparing training on purely synthetic data).
This answer comes from the articleGraphGen: Fine-tuning Language Models Using Knowledge Graphs to Generate Synthetic DataThe
May not be reproduced without permission:AI productivity tools " How to avoid model overfitting problems caused by synthetic data?
