# Generators in Python: A Complete Guide

## **Introduction to Generators**

> Generators in Python are a special type of iterable that allow **lazy evaluation**, meaning they generate values one at a time **instead of storing them all in memory at once**. They are useful for handling large datasets, optimizing memory usage, and improving performance in scenarios where iterating over huge amounts of data is required.

## **Why Use Generators?**

* **Improves Memory Efficiency:** Unlike lists, generators do not store all values in memory.
    
* **Faster Execution:** Values are generated as needed, reducing unnecessary processing.
    
* **Supports Infinite Sequences:** You can define generators that run indefinitely.
    
* **Simplifies Complex Iteration Logic:** Avoids using multiple temporary variables.
    

---

## **Creating a Basic Generator**

A **generator function** uses the `yield` keyword to return values **lazily**.

### **Example 1: Simple Generator**

```python
def simple_generator():
    yield 1
    yield 2
    yield 3

gen = simple_generator()

print(next(gen))  # Output: 1
print(next(gen))  # Output: 2
print(next(gen))  # Output: 3
```

### **How It Works**

* The function `simple_generator()` does **not** return all values at once.
    
* It **pauses execution** at each `yield` and resumes when `next()` is called.
    
* Once all values are yielded, further `next()` calls raise `StopIteration`.
    

## **Iterating Over a Generator Using** `for` Loop

A generator can be **iterated over** using a `for` loop:

```python
def number_generator():
    for i in range(5):
        yield i

for num in number_generator():
    print(num)
```

### **Output**

```plaintext
0
1
2
3
4
```

This avoids calling `next()` manually and ensures graceful handling of `StopIteration`.

## **Generator Expressions**

Generators can be created using **generator expressions**, similar to list comprehensions.

```python
gen_exp = (x * x for x in range(5))

print(next(gen_exp))  # Output: 0
print(next(gen_exp))  # Output: 1
print(next(gen_exp))  # Output: 4
```

This is **more memory-efficient** than list comprehensions (`[x * x for x in range(5)]`), which store all values in memory.

## **Generators vs Lists (Memory Usage)**

Consider generating **1 million numbers**:

### **Using Lists (Consumes More Memory)**

```python
nums_list = [x * x for x in range(1000000)]
```

### **Using Generators (Efficient Memory Usage)**

```python
nums_gen = (x * x for x in range(1000000))
```

* Lists store **all values** in memory upfront.
    
* Generators **compute values only when needed**, reducing memory footprint.
    

## **Generators for Infinite Sequences**

Generators can be used to create **infinite sequences**:

```python
def infinite_counter():
    num = 0
    while True:
        yield num
        num += 1

counter = infinite_counter()

print(next(counter))  # Output: 0
print(next(counter))  # Output: 1
print(next(counter))  # Output: 2
```

This is ideal for streaming data, real-time processing, or dynamically generating content.

## **Using** `yield` with `send()`

Generators can receive values using the `send()` method.

```python
def greeter():
    name = yield "Enter your name:"
    yield f"Hello, {name}!"

g = greeter()
print(next(g))       # Output: Enter your name:
print(g.send("Rahul"))  # Output: Hello, Rahul!
```

* `next()` starts the generator.
    
* `send()` sends a value to the `yield` expression.
    

## **Using** `yield from` (Delegating Iteration)

To **yield values from another iterable**, use `yield from`:

```python
def sub_generator():
    yield from [10, 20, 30]

for num in sub_generator():
    print(num)
```

### **Output**

```plaintext
10
20
30
```

This is cleaner than looping through another iterable manually.

## **Real-World Applications of Generators**

### **1\. Processing Large Files Without Loading into Memory**

```python
def read_large_file(file_path):
    with open(file_path, "r") as file:
        for line in file:
            yield line

for line in read_large_file("data.txt"):
    print(line)
```

* Processes **huge files efficiently** without reading the entire file at once.
    

### **2\. Streaming Live Sensor Data**

```python
import random
import time

def sensor_stream():
    while True:
        yield random.uniform(20, 30)  # Simulating temperature sensor
        time.sleep(1)  # Wait for next reading

sensor = sensor_stream()

for _ in range(5):  
    print(next(sensor))
```

* Continuously **streams live data** without storing previous values.
    

### **3\. Generating Fibonacci Sequence**

```python
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib_gen = fibonacci()

for _ in range(10):  
    print(next(fib_gen))
```

* Generates Fibonacci numbers **without storing** previous values.