|
| | | | |
| | | |
| Stat |
Members: 3669
Articles: 2'599'751
Articles rated: 2609
24 March 2025 |
|
| | | | |
|
Article overview
| |
|
|
Federated Topic Model and Model Pruning Based on Variational Autoencoder | | Chengjie Ma
; Yawen Li
; Meiyu Liang
; Ang Li
; | | Date: |
1 Nov 2023 | | Abstract: | Topic modeling has emerged as a valuable tool for discovering patterns and
topics within large collections of documents. However, when cross-analysis
involves multiple parties, data privacy becomes a critical concern. Federated
topic modeling has been developed to address this issue, allowing multiple
parties to jointly train models while protecting pri-vacy. However, there are
communication and performance challenges in the federated sce-nario. In order
to solve the above problems, this paper proposes a method to establish a
federated topic model while ensuring the privacy of each node, and use neural
network model pruning to accelerate the model, where the client periodically
sends the model neu-ron cumulative gradients and model weights to the server,
and the server prunes the model. To address different requirements, two
different methods are proposed to determine the model pruning rate. The first
method involves slow pruning throughout the entire model training process,
which has limited acceleration effect on the model training process, but can
ensure that the pruned model achieves higher accuracy. This can significantly
reduce the model inference time during the inference process. The second
strategy is to quickly reach the target pruning rate in the early stage of
model training in order to accelerate the model training speed, and then
continue to train the model with a smaller model size after reaching the target
pruning rate. This approach may lose more useful information but can complete
the model training faster. Experimental results show that the federated topic
model pruning based on the variational autoencoder proposed in this paper can
greatly accelerate the model training speed while ensuring the model’s
performance. | | Source: | arXiv, 2311.00314 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER