lm human preferences

We have hosted the application lm human preferences in order to run this application in our online workstations with Wine or directly.

Quick description about lm human preferences:

lm-human-preferences is the official OpenAI codebase that implements the method from the paper Fine-Tuning Language Models from Human Preferences. Its purpose is to show how to align language models with human judgments by training a reward model from human comparisons and then fine-tuning a policy model using that reward signal. The repository includes scripts to train the reward model (learning to rank or score pairs of outputs), and to fine-tune a policy (a language model) with reinforcement learning (or related techniques) guided by that reward model. The code is provided �as is� and explicitly says it may no longer run out-of-the-box due to dependencies or dataset migrations. It was tested on the smallest GPT-2 (124M parameters) under a specific environment (TensorFlow 1.x, specific CUDA / cuDNN combinations). It includes utilities for launching experiments, sampling from policies, and simple experiment orchestration.

Features:

• Training a reward model from human preference comparisons
• Fine-tuning a policy (language model) guided by the reward model
• Sampling / inference utilities to generate outputs from the trained policy
• Experiment orchestration (launch.py) to combine stages (reward + policy)
• Label handling and mapping from human comparisons to scalar reward signals
• Support for small GPT-2 (124M) model as reference environment

Programming Language: Python.
Categories: