KiRorY
算法分析笔记:贪心算法

Algorithm Analysis Notes: Greedy Algorithm

Greedy Algorithm

1. Basic Idea of the Algorithm

Dynamic programming can be used when optimal substructure is present, but sometimes there are simpler and more efficient methods. A greedy algorithm always makes the choice that looks best at the current moment when solving subproblems. It can be said that its approach is myopic and local; it does not consider the overall optimal solution, focusing only on the solution to the local subproblem.

From the description above, greedy algorithms are more direct and efficient, but their limitations are also significant: for many problems, the overall optimal solution cannot be obtained simply by solving the subproblems optimally. Although greedy algorithms clearly cannot solve all problems, we can still use them to obtain locally optimal solutions.

1.1 Greedy Choice Property

As mentioned above, a greedy algorithm only makes the optimal choice for the current subproblem. Its choice can depend on choices made so far, but it definitely never depends on future choices or the solutions to subproblems. Therefore, this algorithm typically follows a top-down strategy.

1.2 Difference Between Greedy Algorithms and Dynamic Programming

Both algorithms exhibit the optimal substructure property. In the author's previous post Algorithm Analysis Notes: Dynamic Programming, the 0-1 Knapsack Problem was mentioned. A problem similar to this is the Fractional Knapsack Problem (often referred to simply as the Knapsack Problem).

Fractional Knapsack Problem

Similar to the 0-1 knapsack problem, given \(n\) items, item \(i\) has weight \(w_i\), value \(v_i\), and the knapsack capacity is \(c\). The goal is to choose items to put into the knapsack such that the total value of items in the knapsack is maximized. However, in the fractional knapsack problem, you can take a fraction of item \(i\) rather than having to take the item as a whole. Therefore, the fractional knapsack problem can be loaded directly based on the value per unit weight of each item. Items with higher value per unit weight are prioritized; once an item is fully packed, if the capacity \(c\) is still not reached, the item with the next highest unit value is chosen, continuing until the knapsack is full. Obviously, this problem can be solved using a greedy algorithm, whereas the 0-1 knapsack problem cannot—because items in the 0-1 knapsack problem cannot be divided, which means the overall optimal solution does not necessarily correspond to the top-down optimal choices of subproblems. The basic code of the algorithm is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
void knapsack(int a, float M, float v[], float w[], float x[])
{
Sort(n,v,w);
int i;
for (i = 1; i <= n; i++)
{
x[i] = 0;
}
float c = M;
for (i = 1;i <= n; i++)
{
if (w[i] > c)
{
break;
}
x[i] = 1;
c -= w[i];
}

if(i <= n)
{
x[i] = c/w[i];
}
}

The upper bound of the computation time for the algorithm above is \(O(n \log{n})\).

2. Algorithm Applications

2.1 Single Source Shortest Path

2.1.1 Problem Description

Given a weighted directed graph \(G=(V,E)\), where each edge's weight is a non-negative real number, and a given vertex in \(V\) is designated as the source. Find the shortest path from the source to all other vertices in \(V\). The diagram below illustrates this problem:

2.1.2 Dijkstra's Algorithm

Dijkstra's algorithm is another classic example of the greedy algorithm design strategy. It selects the vertex \(u\) in \(V-S\) with the shortest distance and adds it to \(S\). Then, for each vertex \(v\) in \(V-S\), if there exists an edge from \(u\) to \(v\), it updates the shortest path from the source to \(v\).

We describe Dijkstra's algorithm as follows: given an input weighted directed graph \(G=(V,E), V=\left\{1,2,\cdots,n\right\}\), with vertex \(v\) as the source. A 2D array \(c[i][j]\) is used to represent the weight of edge \((i,j)\). An array \(dist[i]\) is used to store the current shortest path length from the source to vertex \(i\). The basic code is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
template <class Type>
void Dijkstra(int n, int v, Type dist[], int prev[], Type **c)
{
bool s[maxint]; // Determine whether the shortest path has been found
for (int i = 1; i <= n; i++)
{
dist[i] = c[v][i];
s[i] = false;
if (dist[i] == maxint)
{
prev[i] = 0;
}
else
{
prev[i] = v;
}
}
dist[v] = 0;
s[v] = true;
for (int i = 1; i < n; i++)
{
Type temp = maxint;
int u = v;
for (int j = 1; j <= n; j++)
{
if ((!s[j]) && (dist[j] < temp))
{
u = j;
temp = dist[j];
}
}
s[u] = true;
for (int j = 1; j <= n; j++)
{
if ((!s[j]) && (c[u][j] < maxint))
{
Type newdist = dist[u] + c[u][j];
if (newdist < dist[j])
{
dist[j] = newdist;
prev[j] = u;
}
}
}
}
}

The overall time required to complete the algorithm above is \(O(n^2)\).

For the directed graph given in problem description 2.1.1 above, we compute the shortest paths from source vertex 1 to other vertices and trace its iteration process, recorded in the table below:

Iteration S u dist[2] dist[3] dist[4] dist[5]
Initial {1} -- 10 maxint 30 100
1 {1,2} 2 10 60 30 100
2 {1,2,4} 4 10 50 30 90
3 {1,2,4,3} 3 10 50 30 60
4 {1,2,4,3,5} 5 10 50 30 60

For a detailed step-by-step analysis of the algorithm above, refer to this article: Dijkstra's Shortest Path Algorithm - A Visual Introduction

2.2 Minimum Spanning Tree

2.2.1 Problem Description

Let \(G(V,E)\) be an undirected connected weighted graph. The weight of each edge \((v,w)\) in \(E\) is \(c[v][w]\). If a subgraph \(G'\) of \(G\) is a tree containing all vertices of \(G\), then \(G'\) is called a spanning tree of \(G\). Among these, the spanning tree with the minimum total cost (or the minimum total edge weight) is called the Minimum Spanning Tree.

Before solving this problem, we need to understand its properties. Let \(G=(V,E)\) be a connected weighted graph, and let \(U\) be a proper subset of \(V\). If \((u,v)\in E\), \(u\in U\), \(v \in V-U\), and \((u,v)\) has the minimum weight among all such edges crossing the cut, then this edge must belong to a minimum spanning tree of \(G\).

2.2.2 Prim's Algorithm

For the aforementioned \(G\), first set \(S=\left\{1\right\}\). As long as \(S\) is a proper subset of \(V\), make the following choice: pick the edge \((i,j)\) with minimum \(c[i][j]\) satisfying \(i \in S\) and \(j \in V-S\), and add vertex \(j\) to \(S\). Repeat until \(S=V\). In simple terms, it involves iteratively adding the "shortest" connecting edge to the spanning tree until all vertices are included, thereby obtaining the minimum spanning tree. Take the following connected weighted graph as an example:

The image below illustrates the process of Prim's algorithm:

Simply put, the algorithm takes the original graph apart, starts from the initial vertex, evaluates all edges connecting visited vertices to unvisited vertices, finds the shortest one, adds the corresponding unvisited vertex to the set of visited vertices, and repeats this process. Below is the basic code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
template <class Type>
void Prim(int n, Type** c)
{
Type lowcost[maxint];
int closest[maxint];
bool s[maxint];
s[1] = true;
for (int i = 2; i <= n; i++)
{
lowcost[i] = c[1][i];
closest[i] = 1;
s[i] = false;
}

for(int i = 1; i < n; i++)
{
Type min = inf;
int j = 1;
for(int k = 2; k <= n; k++)
{
if((lowcost[k] < min) && (!s[k]))
{
min = lowcost[k];
j = k;
}
}
}

cout << "(" << closest[j] << "," << j << ")" << endl;
s[j] = ture;
}

Its time complexity is \(O(n^2)\).

2.2.3 Kruskal's Algorithm

Kruskal's algorithm is another commonly used algorithm for solving the minimum spanning tree problem. If Prim's algorithm starts from adjacent vertices and finds the next vertex by picking the shortest edge, Kruskal's algorithm starts by finding the shortest edges, using edges as the starting point to connect vertices and build the minimum spanning tree.

The basic idea of the algorithm is: initially treat each vertex in the graph as an independent component (or tree). Sort all edges in non-decreasing order of weight, and inspect each edge in sequence to see whether it connects two independent components. If it does, add this edge and the connected vertices to the tree; otherwise, skip it and inspect the next edge. Once all edges have been processed, the minimum spanning tree is obtained.

Below is the basic code for the algorithm:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
template <class Type>

class EdgeNode
{
freind ostream& operator<< (ostream&,EdgeNode<Type>);
friend bool Kruskal(int, int, EdgeNode<Type>*, EdgeNode<Type>* );
friend void main(void);
public:
operator Type() const
{
return weight;
}
private:
Type weight;
int u, v;
}

//Kruskal algorithm
template <class Type>
bool Kruskal(int n, int e, EdgeNode<Type> E[], EdgeNode<Type> t[])
{
MinHeap<EdgeNode<Type>> H(1); // Use a min-heap to store sorted edges
H.Initialize(E, e, e);
UnionFind UF(n); // This is an abstract disjoint-set data structure supporting union and find operations
int k = 0;
while (e && k < n-1)
{
EdgeNode<int> x;
H.DeleteMin(x);
e--;
int a = U.Find(x.u);// Find operation, returns the representative/root of the component containing the vertex
int b = U.Find(x.v);
if (a != b)// Check if vertices are in the same component
{
t[k++] = x;
U.Union(a, b);// Union operation, merges the two components
}
H.Deactivate(x);
return (k == n-1);
}
}

For a graph with \(e\) edges, constructing the priority queue takes \(O(e)\) time. DeleteMin takes \(O(\log{e})\) time. Implementing UnionFind requires \(O(e \log{e})\) or \(O(e \alpha(e))\) time (where \(\alpha\) is the inverse Ackermann function, often written as \(O(e \log^*{e})\)). Therefore, the overall time complexity of Kruskal's algorithm is \(O(e \log{e})\).

From this, it can be seen that when \(e=\Omega(n^2)\) (dense graph), Kruskal's algorithm performs worse than Prim's algorithm, but when \(e=o(n^2)\) (sparse graph), Kruskal's algorithm performs better than Prim's algorithm.

Author:KiRorY
Permalink:https://kirory.xyz/posts/6/
License:This work is licensed under CC BY-NC-SA 3.0 CN