Training in progress, step 3500

README.md

@@ -1,223 +1,55 @@
----
-library_name: transformers
-language:
-- en
-tags:
-- generated_from_trainer
-datasets:
-- maliced/l2-arctic
-model-index:
-- name: MDD Transformer Tiny
-  results: []
----
-
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# MDD Transformer Tiny
-
-This model is a fine-tuned version of [](https://huggingface.co/) on the L2 Arctic dataset.
-It achieves the following results on the evaluation set:
-- Loss: 2.7396
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 1e-05
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 500
-- training_steps: 4000
-- mixed_precision_training: Native AMP
-
-### Training results
-
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 3.9162        | 0.0199 | 25   | 3.9053          |
-| 3.9087        | 0.0398 | 50   | 3.8977          |
-| 3.8963        | 0.0598 | 75   | 3.8853          |
-| 3.8862        | 0.0797 | 100  | 3.8683          |
-| 3.867         | 0.0996 | 125  | 3.8470          |
-| 3.8441        | 0.1195 | 150  | 3.8220          |
-| 3.8197        | 0.1394 | 175  | 3.7936          |
-| 3.7952        | 0.1594 | 200  | 3.7623          |
-| 3.755         | 0.1793 | 225  | 3.7280          |
-| 3.7265        | 0.1992 | 250  | 3.6916          |
-| 3.695         | 0.2191 | 275  | 3.6536          |
-| 3.6635        | 0.2390 | 300  | 3.6135          |
-| 3.62          | 0.2590 | 325  | 3.5717          |
-| 3.5774        | 0.2789 | 350  | 3.5281          |
-| 3.5498        | 0.2988 | 375  | 3.4830          |
-| 3.4899        | 0.3187 | 400  | 3.4364          |
-| 3.4769        | 0.3386 | 425  | 3.3878          |
-| 3.4147        | 0.3586 | 450  | 3.3388          |
-| 3.377         | 0.3785 | 475  | 3.2900          |
-| 3.3185        | 0.3984 | 500  | 3.2455          |
-| 3.3012        | 0.4183 | 525  | 3.2052          |
-| 3.264         | 0.4382 | 550  | 3.1712          |
-| 3.203         | 0.4582 | 575  | 3.1432          |
-| 3.188         | 0.4781 | 600  | 3.1190          |
-| 3.1786        | 0.4980 | 625  | 3.0983          |
-| 3.1699        | 0.5179 | 650  | 3.0802          |
-| 3.1318        | 0.5378 | 675  | 3.0647          |
-| 3.1235        | 0.5578 | 700  | 3.0516          |
-| 3.101         | 0.5777 | 725  | 3.0393          |
-| 3.0758        | 0.5976 | 750  | 3.0277          |
-| 3.1111        | 0.6175 | 775  | 3.0175          |
-| 3.101         | 0.6375 | 800  | 3.0075          |
-| 3.0834        | 0.6574 | 825  | 2.9986          |
-| 3.0237        | 0.6773 | 850  | 2.9908          |
-| 3.0341        | 0.6972 | 875  | 2.9833          |
-| 3.0629        | 0.7171 | 900  | 2.9757          |
-| 3.0283        | 0.7371 | 925  | 2.9693          |
-| 2.9794        | 0.7570 | 950  | 2.9626          |
-| 3.0241        | 0.7769 | 975  | 2.9566          |
-| 3.0006        | 0.7968 | 1000 | 2.9509          |
-| 2.9883        | 0.8167 | 1025 | 2.9452          |
-| 2.9556        | 0.8367 | 1050 | 2.9405          |
-| 2.9768        | 0.8566 | 1075 | 2.9350          |
-| 2.9641        | 0.8765 | 1100 | 2.9300          |
-| 2.9836        | 0.8964 | 1125 | 2.9251          |
-| 2.9594        | 0.9163 | 1150 | 2.9206          |
-| 2.9328        | 0.9363 | 1175 | 2.9160          |
-| 2.967         | 0.9562 | 1200 | 2.9121          |
-| 2.9729        | 0.9761 | 1225 | 2.9075          |
-| 2.9495        | 0.9960 | 1250 | 2.9039          |
-| 2.9256        | 1.0159 | 1275 | 2.8999          |
-| 2.9329        | 1.0359 | 1300 | 2.8959          |
-| 2.9471        | 1.0558 | 1325 | 2.8924          |
-| 2.9211        | 1.0757 | 1350 | 2.8885          |
-| 2.9457        | 1.0956 | 1375 | 2.8850          |
-| 2.8959        | 1.1155 | 1400 | 2.8816          |
-| 2.917         | 1.1355 | 1425 | 2.8777          |
-| 2.9581        | 1.1554 | 1450 | 2.8745          |
-| 2.892         | 1.1753 | 1475 | 2.8711          |
-| 2.9026        | 1.1952 | 1500 | 2.8676          |
-| 2.8666        | 1.2151 | 1525 | 2.8647          |
-| 2.8683        | 1.2351 | 1550 | 2.8614          |
-| 2.8808        | 1.2550 | 1575 | 2.8582          |
-| 2.8811        | 1.2749 | 1600 | 2.8552          |
-| 2.9028        | 1.2948 | 1625 | 2.8520          |
-| 2.9027        | 1.3147 | 1650 | 2.8489          |
-| 2.8698        | 1.3347 | 1675 | 2.8459          |
-| 2.8509        | 1.3546 | 1700 | 2.8429          |
-| 2.8765        | 1.3745 | 1725 | 2.8398          |
-| 2.9026        | 1.3944 | 1750 | 2.8371          |
-| 2.8643        | 1.4143 | 1775 | 2.8344          |
-| 2.8731        | 1.4343 | 1800 | 2.8315          |
-| 2.8199        | 1.4542 | 1825 | 2.8288          |
-| 2.8492        | 1.4741 | 1850 | 2.8262          |
-| 2.8163        | 1.4940 | 1875 | 2.8236          |
-| 2.9043        | 1.5139 | 1900 | 2.8210          |
-| 2.8232        | 1.5339 | 1925 | 2.8186          |
-| 2.8258        | 1.5538 | 1950 | 2.8164          |
-| 2.8085        | 1.5737 | 1975 | 2.8141          |
-| 2.8648        | 1.5936 | 2000 | 2.8120          |
-| 2.8648        | 1.6135 | 2025 | 2.8097          |
-| 2.8369        | 1.6335 | 2050 | 2.8075          |
-| 2.8402        | 1.6534 | 2075 | 2.8057          |
-| 2.8694        | 1.6733 | 2100 | 2.8037          |
-| 2.8073        | 1.6932 | 2125 | 2.8019          |
-| 2.8293        | 1.7131 | 2150 | 2.8001          |
-| 2.803         | 1.7331 | 2175 | 2.7980          |
-| 2.8113        | 1.7530 | 2200 | 2.7961          |
-| 2.8499        | 1.7729 | 2225 | 2.7944          |
-| 2.8048        | 1.7928 | 2250 | 2.7927          |
-| 2.832         | 1.8127 | 2275 | 2.7912          |
-| 2.809         | 1.8327 | 2300 | 2.7894          |
-| 2.8517        | 1.8526 | 2325 | 2.7879          |
-| 2.8059        | 1.8725 | 2350 | 2.7863          |
-| 2.7564        | 1.8924 | 2375 | 2.7849          |
-| 2.8427        | 1.9124 | 2400 | 2.7834          |
-| 2.7905        | 1.9323 | 2425 | 2.7820          |
-| 2.7755        | 1.9522 | 2450 | 2.7805          |
-| 2.7935        | 1.9721 | 2475 | 2.7790          |
-| 2.8234        | 1.9920 | 2500 | 2.7777          |
-| 2.8659        | 2.0120 | 2525 | 2.7764          |
-| 2.7873        | 2.0319 | 2550 | 2.7751          |
-| 2.8081        | 2.0518 | 2575 | 2.7739          |
-| 2.82          | 2.0717 | 2600 | 2.7726          |
-| 2.8274        | 2.0916 | 2625 | 2.7715          |
-| 2.7901        | 2.1116 | 2650 | 2.7705          |
-| 2.7661        | 2.1315 | 2675 | 2.7692          |
-| 2.777         | 2.1514 | 2700 | 2.7679          |
-| 2.8142        | 2.1713 | 2725 | 2.7667          |
-| 2.8137        | 2.1912 | 2750 | 2.7658          |
-| 2.7921        | 2.2112 | 2775 | 2.7648          |
-| 2.7432        | 2.2311 | 2800 | 2.7637          |
-| 2.7788        | 2.2510 | 2825 | 2.7626          |
-| 2.7904        | 2.2709 | 2850 | 2.7616          |
-| 2.7674        | 2.2908 | 2875 | 2.7606          |
-| 2.7698        | 2.3108 | 2900 | 2.7596          |
-| 2.7987        | 2.3307 | 2925 | 2.7588          |
-| 2.7801        | 2.3506 | 2950 | 2.7579          |
-| 2.7838        | 2.3705 | 2975 | 2.7570          |
-| 2.7811        | 2.3904 | 3000 | 2.7561          |
-| 2.7864        | 2.4104 | 3025 | 2.7554          |
-| 2.8022        | 2.4303 | 3050 | 2.7545          |
-| 2.7476        | 2.4502 | 3075 | 2.7535          |
-| 2.7964        | 2.4701 | 3100 | 2.7527          |
-| 2.7696        | 2.4900 | 3125 | 2.7522          |
-| 2.8205        | 2.5100 | 3150 | 2.7515          |
-| 2.7584        | 2.5299 | 3175 | 2.7508          |
-| 2.7627        | 2.5498 | 3200 | 2.7502          |
-| 2.7485        | 2.5697 | 3225 | 2.7495          |
-| 2.749         | 2.5896 | 3250 | 2.7490          |
-| 2.7788        | 2.6096 | 3275 | 2.7484          |
-| 2.7639        | 2.6295 | 3300 | 2.7478          |
-| 2.7427        | 2.6494 | 3325 | 2.7472          |
-| 2.7432        | 2.6693 | 3350 | 2.7467          |
-| 2.8058        | 2.6892 | 3375 | 2.7462          |
-| 2.7592        | 2.7092 | 3400 | 2.7457          |
-| 2.7299        | 2.7291 | 3425 | 2.7452          |
-| 2.7342        | 2.7490 | 3450 | 2.7447          |
-| 2.7837        | 2.7689 | 3475 | 2.7442          |
-| 2.767         | 2.7888 | 3500 | 2.7438          |
-| 2.7443        | 2.8088 | 3525 | 2.7434          |
-| 2.7535        | 2.8287 | 3550 | 2.7429          |
-| 2.7426        | 2.8486 | 3575 | 2.7425          |
-| 2.7515        | 2.8685 | 3600 | 2.7422          |
-| 2.7643        | 2.8884 | 3625 | 2.7419          |
-| 2.7729        | 2.9084 | 3650 | 2.7416          |
-| 2.7254        | 2.9283 | 3675 | 2.7413          |
-| 2.7428        | 2.9482 | 3700 | 2.7411          |
-| 2.7475        | 2.9681 | 3725 | 2.7408          |
-| 2.7293        | 2.9880 | 3750 | 2.7406          |
-| 2.8002        | 3.0080 | 3775 | 2.7404          |
-| 2.7434        | 3.0279 | 3800 | 2.7403          |
-| 2.7217        | 3.0478 | 3825 | 2.7401          |
-| 2.7748        | 3.0677 | 3850 | 2.7400          |
-| 2.7278        | 3.0876 | 3875 | 2.7399          |
-| 2.7534        | 3.1076 | 3900 | 2.7398          |
-| 2.7662        | 3.1275 | 3925 | 2.7397          |
-| 2.7514        | 3.1474 | 3950 | 2.7397          |
-| 2.7905        | 3.1673 | 3975 | 2.7396          |
-| 2.7737        | 3.1873 | 4000 | 2.7396          |
-
-
-### Framework versions
-
-- Transformers 4.52.3
-- Pytorch 2.7.0+cpu
-- Datasets 3.6.0
-- Tokenizers 0.21.1
+---
+library_name: transformers
+language:
+- en
+tags:
+- generated_from_trainer
+datasets:
+- maliced/l2-arctic
+model-index:
+- name: MDD Transformer Tiny
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# MDD Transformer Tiny
+
+This model is a fine-tuned version of [](https://huggingface.co/) on the L2 Arctic dataset.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 1
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- training_steps: 4000
+- mixed_precision_training: Native AMP
+
+### Framework versions
+
+- Transformers 4.52.1
+- Pytorch 2.7.0+cpu
+- Datasets 3.6.0
+- Tokenizers 0.21.1

config.json

@@ -1,65 +1,65 @@
-{
-  "architectures": [
-    "MDDTransformer"
-  ],
-  "batch_first": true,
-  "d_model": 32,
-  "dim_feedforward": 256,
-  "hidden_dropout_prob": 0.1,
-  "input_channels": 2,
-  "max_source_positions": 3000,
-  "model_type": "mdd_transformer",
-  "num_attention_heads_decoder": 2,
-  "num_attention_heads_encoder": 4,
-  "num_classes": 43,
-  "num_cross_attention_heads": 2,
-  "num_decoder_layers": 2,
-  "num_encoder_layers": 2,
-  "num_mel_bins": 80,
-  "projector_activation": "softmax",
-  "tokens": [
-    "SIL",
-    "AA",
-    "AE",
-    "AH",
-    "AO",
-    "AW",
-    "AX",
-    "AY",
-    "B",
-    "CH",
-    "D",
-    "DH",
-    "EH",
-    "ER",
-    "EY",
-    "F",
-    "G",
-    "HH",
-    "IH",
-    "IY",
-    "JH",
-    "K",
-    "L",
-    "M",
-    "N",
-    "NG",
-    "OW",
-    "OY",
-    "P",
-    "R",
-    "S",
-    "SH",
-    "T",
-    "TH",
-    "UH",
-    "UW",
-    "V",
-    "W",
-    "Y",
-    "Z",
-    "ZH"
-  ],
-  "torch_dtype": "float32",
-  "transformers_version": "4.52.3"
-}
+{
+  "architectures": [
+    "MDDTransformer"
+  ],
+  "batch_first": true,
+  "d_model": 32,
+  "dim_feedforward": 256,
+  "hidden_dropout_prob": 0.1,
+  "input_channels": 2,
+  "max_source_positions": 3000,
+  "model_type": "mdd_transformer",
+  "num_attention_heads_decoder": 2,
+  "num_attention_heads_encoder": 4,
+  "num_classes": 43,
+  "num_cross_attention_heads": 2,
+  "num_decoder_layers": 2,
+  "num_encoder_layers": 2,
+  "num_mel_bins": 80,
+  "projector_activation": "softmax",
+  "tokens": [
+    "SIL",
+    "AA",
+    "AE",
+    "AH",
+    "AO",
+    "AW",
+    "AX",
+    "AY",
+    "B",
+    "CH",
+    "D",
+    "DH",
+    "EH",
+    "ER",
+    "EY",
+    "F",
+    "G",
+    "HH",
+    "IH",
+    "IY",
+    "JH",
+    "K",
+    "L",
+    "M",
+    "N",
+    "NG",
+    "OW",
+    "OY",
+    "P",
+    "R",
+    "S",
+    "SH",
+    "T",
+    "TH",
+    "UH",
+    "UW",
+    "V",
+    "W",
+    "Y",
+    "Z",
+    "ZH"
+  ],
+  "torch_dtype": "float32",
+  "transformers_version": "4.52.1"
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b6c64c166dcbfb299718b9885ea6113e229b2b72f73468cf8daa35824d16f9a8
+oid sha256:753071d13f03c5b0f7d0efbb6c6415b7a7d39fc79bfd206c436c9e819e3e2998
 size 1009168

runs/May30_16-09-27_malice-laptop/events.out.tfevents.1748614169.malice-laptop.9668.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3cc29015bbf985928f32e0111286a99b08ed0f23a664cb5790f098fc4682a43f
-size 63143
+oid sha256:2976c41dd3688ea18b3d4874b5045caa97aa0fb72a2fc739b6a0f6e574e11751
+size 72783

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf82f87052e26ded05e89558f5bd204fdbae5947b5556fbda36f3aec8b45dc45
+oid sha256:c1a20204c0ce3ff99c686cea7ac55c8099195049bc2c55c54b6423d4179bf1dc
 size 5905