Publication Library
Leakage-Robust Bayesian Persuasion
Description: We introduce the concept of leakage-robust Bayesian persuasion. Situated between public persuasion [KG11, CCG23, Xu20] and private persuasion [AB19], leakage-robust persuasion considers a setting where one or more signals privately sent by a sender to the receivers may be leaked. We study the design of leakage-robust persuasion schemes and quantify the price of robustness using two formalisms: - The first notion, k-worst-case persuasiveness, requires a scheme to remain persuasive as long as each receiver observes at most k leaked signals. We quantify the Price of Worst-case Robustness (PoWRk) -- i.e., the gap in sender's utility as compared to the optimal private scheme -- as Θ(min{2k,n}) for supermodular sender utilities and Θ(k) for submodular or XOS utilities, where n is the number of receivers. This result also establishes that in some instances, Θ(logk) leakages are sufficient for the utility of the optimal leakage-robust persuasion to degenerate to that of public persuasion. - The second notion, expected downstream utility robustness, relaxes the persuasiveness and considers the impact on sender's utility when receivers best respond to their observations. By quantifying the Price of Downstream Robustness (PoDR) as the gap between the sender's expected utility over random leakage patterns as compared to private persuasion, we show that over several natural and structured distributions of leakage patterns, PoDR improves PoWR to Θ(k) or even Θ(1), where k is the maximum number of leaked signals observable to each receiver across leakage patterns in the distribution. En route to these results, we show that subsampling and masking are general-purpose algorithmic paradigms for transforming private persuasion signaling schemes to leakage-robust ones, with minmax optimal loss in the sender's utility.
Created At: 02 December 2024
Updated At: 02 December 2024
Leveraging Large Language Models for Institutional Portfolio Management Persona-Based Ensembles
Description: Large language models (LLMs) have demonstrated promising performance in various financial applications, though their potential in complex investment strategies remains underexplored. To address this gap, we investigate how LLMs can predict price movements in stock and bond portfolios using economic indicators, enabling portfolio adjustments akin to those employed by institutional investors. Additionally, we explore the impact of incorporating different personas within LLMs, using an ensemble approach to leverage their diverse predictions. Our findings show that LLM-based strategies, especially when combined with the mode ensemble, outperform the buy-and-hold strategy in terms of Sharpe ratio during periods of rising consumer price index (CPI). However, traditional strategies are more effective during declining CPI trends or sharp market downturns. These results suggest that while LLMs can enhance portfolio management, they may require complementary strategies to optimize performance across varying market conditions.
Created At: 02 December 2024
Updated At: 02 December 2024
GPT-4 Passes the Bar Exam
Description: In this paper, we experimentally evaluate the zero-shot performance of a preliminary version of GPT-4 against prior generations of GPT on the entire Uniform Bar Examination (UBE), including not only the multiple-choice Multistate Bar Examination (MBE), but also the open-ended Multistate Essay Exam (MEE) and Multistate Performance Test (MPT) components. On the MBE, GPT-4 significantly outperforms both human test-takers and prior models, demonstrating a 26% increase over ChatGPT and beating humans in five of seven subject areas. On the MEE and MPT, which have not previously been evaluated by scholars, GPT-4 scores an average of 4.2/6.0 as compared to much lower scores for ChatGPT. Graded across the UBE components, in the manner in which a human tast-taker would be, GPT-4 scores approximately 297 points, significantly in excess of the passing threshold for all UBE jurisdictions. These findings document not just the rapid and remarkable advance of large language model performance generally, but also the potential for such models to support the delivery of legal services in society.
Created At: 18 November 2024
Updated At: 18 November 2024
Towards a Classification of Open-Source ML Models and Datasets for Software Engineering
Description: Background: Open-Source Pre-Trained Models (PTMs) and datasets provide extensive resources for various Machine Learning (ML) tasks, yet these resources lack a classification tailored to Software Engineering (SE) needs. Aims: We apply an SE-oriented classification to PTMs and datasets on a popular open-source ML repository, Hugging Face (HF), and analyze the evolution of PTMs over time. Method: We conducted a repository mining study. We started with a systematically gathered database of PTMs and datasets from the HF API. Our selection was refined by analyzing model and dataset cards and metadata, such as tags, and confirming SE relevance using Gemini 1.5 Pro. All analyses are replicable, with a publicly accessible replication package. Results: The most common SE task among PTMs and datasets is code generation, with a primary focus on software development and limited attention to software management. Popular PTMs and datasets mainly target software development. Among ML tasks, text generation is the most common in SE PTMs and datasets. There has been a marked increase in PTMs for SE since 2023 Q2. Conclusions: This study underscores the need for broader task coverage to enhance the integration of ML within SE practices.
Created At: 18 November 2024
Updated At: 18 November 2024
LLMPhy Complex Physical Reasoning Using Large Language Models and World Models
Description: Physical reasoning is an important skill needed for robotic agents when operating in the real world. However, solving such reasoning problems often involves hypothesizing and reflecting over complex multi-body interactions under the effect of a multitude of physical forces and thus learning all such interactions poses a significant hurdle for state-of-the-art machine learning frameworks, including large language models (LLMs). To study this problem, we propose a new physical reasoning task and a dataset, dubbed TraySim. Our task involves predicting the dynamics of several objects on a tray that is given an external impact -- the domino effect of the ensued object interactions and their dynamics thus offering a challenging yet controlled setup, with the goal of reasoning being to infer the stability of the objects after the impact. To solve this complex physical reasoning task, we present LLMPhy, a zero-shot black-box optimization framework that leverages the physics knowledge and program synthesis abilities of LLMs, and synergizes these abilities with the world models built into modern physics engines. Specifically, LLMPhy uses an LLM to generate code to iteratively estimate the physical hyperparameters of the system (friction, damping, layout, etc.) via an implicit analysis-by-synthesis approach using a (non-differentiable) simulator in the loop and uses the inferred parameters to imagine the dynamics of the scene towards solving the reasoning task. To show the effectiveness of LLMPhy, we present experiments on our TraySim dataset to predict the steady-state poses of the objects. Our results show that the combination of the LLM and the physics engine leads to state-of-the-art zero-shot physical reasoning performance, while demonstrating superior convergence against standard black-box optimization methods and better estimation of the physical parameters.
Created At: 13 November 2024
Updated At: 13 November 2024