The bruteforce approach is to look at all pairs of (i,j) to (k,l) and check if its filled with 1s. This approach however is O(NNNNN^2) = O(N^6). [ N^4 ways to choose i,j,k,l and then N^2 elements in the square ].

We can optimize this approach if we had additional space to store results for your previous calculations. Maybe if we knew the result for (i, j) to (k, l - 1) or (i, j) to (k - 1, l) or both.

We can improve from N^6 by storing in dp[i][j][k][l] if (i,j) to (k,l) is all filled with 1.

dp[i][j[k][l] = 1 iff dp[i][j][k][l-1] = 1 && dp[i][j][k-1][l] = 1 and matrix[k][l] = 1.

Now we can improve this further. What if with every (i,j) we stored the length of 1s in the same row i starting from (i,j).

We can move down in the column j from row i and determine the largest rectangle without having to visit all cells.

Even Better Solution :

Lets dp[i][j] denote the length of 1s in the same row i starting from (i,j).

So our current max with one end of the rectangle at (i,j) would be dp[i][j].

As we move to the next column, there are 2 cases :

1) dp[i+1][j] >= dp[i][j] which means that we can take dp[i][j] 1s from next column as well and extend our current rectangle as it is, with one more extra row.

11100000 - 111

11111100 - 111

2) dp[i+1][j] < dp[i][j] which means that if we want to extend our current rectangle to next row, we need to reduce the number of columns in it to dp[i+1][j]

11100000 - 11

11000000 - 11

As mentioned above, we keep increasing the columns and adjusting the width of the rectangle.

O(N^3) time complexity.

Even though N^3 is acceptable, it might be worth exploring a better solution.

If we notice, laying out dp[i][j] helps us make histograms in every row. Then the problem becomes of finding the maximum area in histograms which can be done in O(n). This would lead to an O(N^2) solution. However, I am posting here is O(N^3) .

// Author :: Gaurav Ahirwar
#include<bits/stdc++.h>
#define R 3
#define C 3
using namespace std;
int solve(char matrix[][C]) {
int rows = R;
if (rows == 0) return 0;
int cols = C;
if (cols == 0) return 0;
int dp[rows][cols];
memset(dp, 0, sizeof dp);
int area = 0;
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
if (matrix[i][j] == '1') {
if (j == 0) dp[i][j] = 1;
else dp[i][j] = dp[i][j - 1] + 1;
int y = 1;
int x = cols;
while((i - y + 1 >= 0) && (matrix[i - y + 1][j] == '1')) {
x = min(x, dp[i - y + 1][j]);
area = max(area, x * y);
y++;
}
}
}
}
return area;
}
int main() {
char matrix[][C] = {{'1', '1', '1'},
{'0', '1', '1'},
{'1', '0', '0'}};
cout << solve(matrix) << endl;
return 0;
}