libauc.optimizers
=========================

An overview of the :obj:`~libauc.optimizers` module is summarized as follow: 

.. list-table::

   * - **Optimizer**
     - **Loss Function**
   * - :obj:`~libauc.optimizers.pesg.PESG`: Proximal Epoch Stochastic Gradient Method (PESG)
     - :obj:`~libauc.losses.auc.AUCMLoss`
   * - :obj:`~libauc.optimizers.pdsca.PDSCA`: Primal-Dual Stochastic Compositional Adaptive (PDSCA)
     - :obj:`~libauc.losses.auc.CompositionalAUCLoss`
   * - :obj:`~libauc.optimizers.soap.SOAP`: Stochastic Optimization of Average Precision (SOAP)
     - :obj:`~libauc.losses.auc.AveragePrecisionLoss`
   * - :obj:`~libauc.optimizers.sopa.SOPA`, :obj:`~libauc.optimizers.sopa\_s.SOPAs`, :obj:`~libauc.optimizers.sota\_s.SOTAs`
     - :obj:`~libauc.losses.auc.pAUCLoss`
   * - :obj:`~libauc.optimizers.sopa.SOPA`: Stochastic Optimization for one-way pAUC (SOPA)
     - :obj:`~libauc.losses.auc.pAUC_CVaR_Loss`
   * - :obj:`~libauc.optimizers.sopa\_s.SOPAs`: Stochastic Optimization for one-way pAUC (SOPAs)
     - :obj:`~libauc.losses.auc.pAUC_DRO_Loss`
   * - :obj:`~libauc.optimizers.sota\_s.SOTAs`: Stochastic Optimization for Two-way pAUC Soft-version (SOTAs) 
     - :obj:`~libauc.losses.auc.tpAUC_KL_Loss`
   * - :obj:`~libauc.optimizers.adamw.Adamw`, :obj:`~libauc.optimizers.adam.Adam`, :obj:`~libauc.optimizers.sgd.SGD`
     - :obj:`~libauc.losses.auc.PairwiseAUCLoss`
   * - :obj:`~libauc.optimizers.pesg.PESG`: Proximal Epoch Stochastic Method (PESG)
     - :obj:`~libauc.losses.auc.MultiLabelAUCMLoss`
   * - :obj:`~libauc.optimizers.soap.SOAP`:  Stochastic Optimization of Average Precision (SOAP)
     - :obj:`~libauc.losses.auc.meanAveragePrecisionLoss`
   * - :obj:`~libauc.optimizers.sopa.SOPA`, :obj:`~libauc.optimizers.sopa\_s.SOAPs`, :obj:`~libauc.optimizers.sota\_s.SOTAs`
     - :obj:`~libauc.losses.auc.MultiLabelpAUCLoss`
   * - :obj:`~libauc.optimizers.sogclr.SogCLR`: Stochastic Optimization for Global Contrastive Learning (SogCLR)
     - :obj:`~libauc.losses.contrastive.GCLoss`
   * - :obj:`~libauc.optimizers.isogclr.iSogCLR`: SogCLR with Automatic Temperature Individualization (iSogCLR)
     - :obj:`~libauc.losses.contrastive.GCLoss`
   * - :obj:`~libauc.optimizers.song.SONG`: Stochastic Optimization for NDCG (SONG)
     - :obj:`~libauc.losses.ranking.NDCGLoss`
   * - :obj:`~libauc.optimizers.song.SONG`: Stochastic Optimization for NDCG (SONG)
     - :obj:`~libauc.losses.ranking.ListwiseCELoss`
       

We also adapted some popular optimizers from PyTorch codebase as follow:

   - :obj:`~libauc.optimizers.adamw.AdamW`: Adam with decoupled weight decay regularization (AdamW)
   - :obj:`~libauc.optimizers.adam.Adam`: Adam
   - :obj:`~libauc.optimizers.sgd.SGD`: Stochastic Gradient Descent (SGD)
   - :obj:`~libauc.optimizers.lars.LARS`: Layer-wise Adaptive Rate Scaling (LARS)


libauc.optimizers.pesg 
-----------------------------

.. automodule:: libauc.optimizers.pesg
   :members:
   :undoc-members:


libauc.optimizers.pdsca
------------------------------

.. automodule:: libauc.optimizers.pdsca
   :members:
   :undoc-members:


libauc.optimizers.soap
-----------------------------

.. automodule:: libauc.optimizers.soap
   :members:
   :undoc-members:


libauc.optimizers.sopa 
-----------------------------

.. automodule:: libauc.optimizers.sopa
   :members:
   :undoc-members:


libauc.optimizers.sopa\_s
--------------------------------

.. automodule:: libauc.optimizers.sopa_s
   :members:
   :undoc-members:


libauc.optimizers.sota\_s
--------------------------------

.. automodule:: libauc.optimizers.sota_s
   :members:
   :undoc-members:

libauc.optimizers.song 
-----------------------------

.. automodule:: libauc.optimizers.song
   :members:
   :undoc-members:


libauc.optimizers.sogclr 
-----------------------------

.. automodule:: libauc.optimizers.sogclr
   :members:
   :undoc-members:

libauc.optimizers.isogclr 
-----------------------------

.. automodule:: libauc.optimizers.isogclr
   :members:
   :undoc-members:

libauc.optimizers.midam 
-----------------------------

.. automodule:: libauc.optimizers.midam
   :members:
   :undoc-members:


libauc.optimizers.lars
-----------------------------

.. automodule:: libauc.optimizers.lars
   :members:
   :undoc-members:

libauc.optimizers.sgd 
----------------------------

.. automodule:: libauc.optimizers.sgd
   :members:
   :undoc-members:

libauc.optimizers.adam
-----------------------------

.. automodule:: libauc.optimizers.adam
   :members:
   :undoc-members:


libauc.optimizers.adamw
------------------------------

.. automodule:: libauc.optimizers.adamw
   :members:
   :undoc-members: