Skip to content

Enhanced Painting Extractor with Selenium WebDriver #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Enhanced Painting Extractor with Selenium WebDriver #325

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9df5aba
first working version
Mujadded Apr 25, 2025
43c8f27
added outputs
Mujadded Apr 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .ruby-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.3.2
9 changes: 9 additions & 0 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
source 'https://rubygems.org'

gem 'nokogiri', '~> 1.15.5'
gem 'json', '~> 2.6.3'
gem 'selenium-webdriver', '~> 4.11.0'
group :development, :test do
gem 'rspec', '~> 3.12.0'
gem 'pry', '~> 0.14.2'
end
49 changes: 49 additions & 0 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
GEM
remote: https://rubygems.org/
specs:
coderay (1.1.3)
diff-lcs (1.6.1)
json (2.6.3)
method_source (1.1.0)
mini_portile2 (2.8.8)
nokogiri (1.15.7)
mini_portile2 (~> 2.8.2)
racc (~> 1.4)
pry (0.14.2)
coderay (~> 1.1)
method_source (~> 1.0)
racc (1.8.1)
rexml (3.4.1)
rspec (3.12.0)
rspec-core (~> 3.12.0)
rspec-expectations (~> 3.12.0)
rspec-mocks (~> 3.12.0)
rspec-core (3.12.3)
rspec-support (~> 3.12.0)
rspec-expectations (3.12.4)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0)
rspec-mocks (3.12.7)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0)
rspec-support (3.12.2)
rubyzip (2.4.1)
selenium-webdriver (4.11.0)
rexml (~> 3.2, >= 3.2.5)
rubyzip (>= 1.2.2, < 3.0)
websocket (~> 1.0)
websocket (1.2.11)

PLATFORMS
arm64-darwin-24
ruby

DEPENDENCIES
json (~> 2.6.3)
nokogiri (~> 1.15.5)
pry (~> 0.14.2)
rspec (~> 3.12.0)
selenium-webdriver (~> 4.11.0)

BUNDLED WITH
2.6.8
6 changes: 6 additions & 0 deletions extract.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env ruby

require_relative 'lib/cli'

# Run the CLI with command line arguments
CLI.run(ARGV)
52 changes: 52 additions & 0 deletions fixtures/leonardo-da-vinci-paintings.html
View file
Open in desktop

Large diffs are not rendered by default.

56 changes: 56 additions & 0 deletions fixtures/pablo-picasso-paintings.html
View file
Open in desktop

Large diffs are not rendered by default.

64 changes: 64 additions & 0 deletions fixtures/sample.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
<!DOCTYPE html>
<html>
<head>
<title>Vincent van Gogh paintings - Google Search</title>
</head>
<body>
<!-- Main carousel container with the Cz5hV class used in the extractor -->
<div class="Cz5hV">
<!-- First painting with iELo6 class for artwork item -->
<div class="iELo6">
<!-- Image container with taFZJe class -->
<!-- Placeholder image with data-src attribute for the real image -->
<img class="taFZJe" src="data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw=="
data-src="https://example.com/starry-night.jpg"
alt="The Starry Night">
<!-- Details container with KHK6lb class -->
<div class="KHK6lb">
<!-- Title with pgNMRc class -->
<div class="pgNMRc">The Starry Night</div>
<!-- Year with cxzHyb class -->
<div class="cxzHyb">1889</div>
</div>
<!-- Link to painting details -->
<a href="/search?q=the+starry+night+van+gogh">View details</a>
</div>

<!-- Second painting -->
<div class="iELo6">
<!-- Regular image with src attribute (no placeholder) -->
<img class="taFZJe" src="https://example.com/sunflowers.jpg"
alt="Sunflowers">
<div class="KHK6lb">
<div class="pgNMRc">Sunflowers</div>
<div class="cxzHyb">1888</div>
</div>
<a href="https://example.com/sunflowers">View details</a>
</div>

<!-- Third painting -->
<div class="iELo6">
<!-- Placeholder image with empty src -->
<img class="taFZJe" src=""
data-src="https://example.com/self-portrait.jpg"
alt="Self-Portrait">
<div class="KHK6lb">
<div class="pgNMRc">Self-Portrait</div>
<div class="cxzHyb">1889</div>
</div>
<a href="/search?q=self+portrait+van+gogh">View details</a>
</div>

<!-- Fourth painting with alternative formatting -->
<div class="iELo6">
<img class="taFZJe" src="https://example.com/potato-eaters.jpg"
alt="The Potato Eaters">
<div class="KHK6lb">
<div class="pgNMRc">The Potato Eaters</div>
<div class="cxzHyb">1885</div>
</div>
<!-- No link element for this painting -->
</div>
</div>
</body>
</html>
Loading