Large Language Models (LLMs) have various capabilities, but their ability to gauge complexity can be nuanced. Here are the key points from the podcast excerpts:

1. Basic Functionality and Limitations:
- LLMs can operate in a space of concepts but struggle with understanding the physical world and visual data. They can process visual representations if fed during training, but these are typically considered "hacks" and not fully integrated capabilities 00:17:00 - 00:18:25.
- They lack persistent memory, the ability to reason, and the ability to plan effectively. These are essential characteristics for gauging complexity, which LLMs can only do in a very primitive way 00:10:00.

2. Auto-regressive Nature and Scaling:
- Auto-regressive LLMs, which predict the next token based on the previous ones, have shown surprising capabilities when scaled up. This scaling allows them to understand more about language, suggesting that they can handle complexity to some extent by processing large amounts of data 01:00:00 - 01:00:24.

3. Hallucinations and Accuracy:
- Despite their capabilities, LLMs can produce hallucinations due to the nature of autoregressive predictions, which can affect their accuracy in gauging complexity accurately 01:13:00 - 01:13:26.

4. Fine-tuning and Specific Tasks:
- LLMs can be fine-tuned for specific tasks, which can help them solve particular problems more accurately. For example, an LLM could potentially handle a problem like determining the travel logistics between New York and Paris if it is fine-tuned accordingly 00:55:01 - 00:55:28.

Overall, while LLMs have shown remarkable capabilities, their ability to gauge complexity is limited by their lack of persistent memory, reasoning, and planning abilities, as well as their tendency to produce hallucinations. However, with sufficient scaling and fine-tuning, they can handle certain complex tasks to a reasonable extent.