So, I ran into a need to scan some files for viruses on Ubuntu this past week. However, a couple of things prevented this from being straight forward:

  • I couldn’t be sure where these files were being stored.

  • I didn’t want to scan the entire filesystem just to get at these few files.

So I went out looking for a way to scan only new files with clamscan. After a good bit of digging, I ran across this old thread where someone had a similar question. So after getting a pointer there, I was able to make this happen. Here is the way to do it:

##Install ClamAV##

  • Issue sudo apt-get install clamav to get the freshclam and clamscan commands.

  • Update the virus definitions with a sudo freshclam. This will take a few seconds the first time.

##Get Ya Find Right##

Now this took a bit of playing around with to get where I wanted. I wanted to find all files of a certain type that had been created or modified in the past week. There’s also some differences to note about mtime vs. ctime vs. atime as a flag for find. Linux-FAQs.info did a good job of explaining these differences:

ctime - ctime is the inode or file change time. The ctime gets updated when the file attributes are changed, like changing the owner, changing the permission or moving the file to an other filesystem but will also be updated when you modify a file.

mtime - mtime is the file modify time. The mtime gets updated when you modify a file. Whenever you update content of a file or save a file the mtime gets updated.

*Most of the times ctime and mtime will be the same, unless only the file attributes are updated. In that case only the ctime gets updated.*

atime - atime is the file access time. The atime gets updated when you open a file but also when a file is used for other operations like grep, sort, cat, head, tail and so on.

For my purposes, I wanted EVERYTHING that had changed, so that pointed to using the ctime flag. For a first test, I just wanted to see how many items find would return. I was able to do that by issuing (edited to look for .md files, just for laughs): sudo find / -name "*.md" -ctime -7 -type f | wc -l. That command simply returns the number 5 on my machine at the time of writing. Let’s not pipe it out to wc so we can see those files:

rsmitty@rsmitty-notebook:~$ sudo find / -name "*.md" -ctime -7 -type f
/home/rsmitty/Desktop/usenet-dashboard/README.md
/home/rsmitty/Desktop/rsmitty.github.io/_posts/2014-09-07-creating-a-blog-2.md
/home/rsmitty/Desktop/rsmitty.github.io/_posts/2014-09-16-clamscan-files-by-date.md
/home/rsmitty/.xbmc/addons/service.xbmc.versioncheck/README.md
/usr/share/xbmc/addons/service.xbmc.versioncheck/README.md

Note: You’ll have to do almost everything with ClamAV as root user. Also, you can change the number of days to scan for by changing the ‘-7’ portion of the command.

Okay, slightly more interesting, and now we know what we’re working with. Let’s move on…

##Enter Xargs##

Using xargs was a first for me as part of this little endeavor. It’s a really handy tool to add to the toolbox! If you’ve ever tried to run a bash command like rm * and received an error like “Argument list too long”, xargs is the answer to your problems. It takes the argument list and breaks it down into smaller pieces. It’ll then run subsequent commands with each sublist. For the purposes of what I was doing initally, there were about 7,000 files to scan, xargs was able to break those up into two scans of ~3,000 files and one scan of ~1,000. Worked great!

Here’s the final command that I used:

sudo find / -name "*.md" -ctime -7 -type f -print0 | sudo xargs -0 clamscan --remove --log=/home/rsmitty/clamscan.log
  • The –remove flag just means that if a vulnerability is found in that file, delete the file immediately.
  • the –log flag sets the path of the log file that clamscan will write. You will need to do this for sure if you have lots of files to scan, because several scan summaries will be written to this file.

After running, the log file will look something like this:

rsmitty@rsmitty-notebook:~$ sudo cat /home/rsmitty/clamscan.log 

----------- SCAN SUMMARY -----------
Known viruses: 3560111
Engine version: 0.98.1
Scanned directories: 0
Scanned files: 5
Infected files: 0
Data scanned: 0.02 MB
Data read: 0.01 MB (ratio 2.50:1)
Time: 6.441 sec (0 m 6 s)

After part one of this tutorial, we have a development environment setup that’s ready for us to really start making some headway in getting a presentable blog up and running. In this tutorial, I’ll show how I plugged Bootstrap’s example blog theme into my site and also guide through creating a first post.

###Create A Default Layout

  • First, we’ll want to get our code syntax highligting to work. Create a CSS file in the root of your Github repo called pygments.css by issuing: pygmentize -S default -f html > pygments.css

Note: This step may not be necessary for everyone, but I’ll be including a call to pygments.css in the next step. So if you’re following along step-by-step, it may be worth going ahead and doing this.

  • Let’s create a default template for all of our pages to use. This is where the special folders that we created in the last tutorial come into play. Change into the _layouts directory and open a file called default.html for editing. I’ve created a basic template that imports the necessary CSS and Javascript files for the Bootstrap example blog. I’m relying on the public URLs for those imports; however, if you want to ensure that they are always functional, you’ll want to download these files manually and import them from a path inside your repo. Add the following to the default.html file:
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
    <meta http-equiv='X-UA-Compatible' content='IE=edge'>
    <meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1.0'>

    <title>{{ site.name }}</title>

    <meta name="author" content="{{ site.name }}" />
    <meta name="description" content="{{ site.description }}">

    <!-- Bootstrap CSS Files -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">
    <link rel="stylesheet" href="http://getbootstrap.com/examples/blog/blog.css">
    
    <!-- Pygments CSS File for Syntax Highlighting -->
    <link rel="stylesheet" href="/pygments.css">

  </head>

  <body>

  <div class="blog-masthead navbar-fixed-top">
      <div class="container">
        <nav class="blog-nav">
          <a class="blog-nav-item" href="#">Home</a>
          <a class="blog-nav-item" href="#">About</a>
        </nav>
      </div>
    </div>

    <div class="container">
      <div class="blog-header">
        <h1 class="blog-title">{{site.name}}</h1>
        <p class="lead blog-description">{{site.description}}</p>
      </div>

    <div class="row">
      <div class="col-sm-8 blog-main">
          {{ content }}
      </div>

    <div class="col-sm-3 col-sm-offset-1 blog-sidebar">
      <div class="sidebar-module sidebar-module-inset">
        <h4>About</h4>
        <p>{{site.about}}</p>
      </div>
      <div class="sidebar-module sidebar-module-inset">
        <h4>Contact Info</h4>
        <ol class="list-unstyled">
          <li><a href="#" target="_blank">Github</a></li>
          <li><a href="#" target="_blank">LinkedIn</a></li>
        </ol>
      </div>
    </div>
  </div>
  </div>

  <!-- Bootstrap JS Files -->
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>

  </body>
</html>

###Edit The Index Page

  • Let’s tell our index.html page to use our snazzy new layout. Edit your index.html page to look like the following:
---
layout: default
---

Hello, World!

Note: The YAML at the top of this page allows us to specify certain behavior from page to page. For example, we’re telling this page to load our default template. However, if we created another template called foo later on in the future, we could simply change the layout variable to point to the new foo layout.

  • Refresh the localhost:4000 page in your browser and you should see the new layout load, along with all of the site info that we entered in _config.yml in the last tutorial.

  • Looks good! Now, we’d rather the index page say something other than “Hello, World”. In fact, we want the first thing that people see to be a list of all of the highly interesting and potentially life changing posts that we’ll be making as our blogging continues. Let’s now edit our index.html to look like the following:

---
layout: default
---

{% for post in site.posts %}
    <div class="blog-post">
      <h2 class="blog-post-title"><a href="">{{ post.title }}</a></h2>
      <p class="blog-post-meta">Written {{ post.date | date: "%m.%d.%Y" }} by {{post.author}}</p>
        {{ post.content | truncatewords:40}}  
      <a href="" class="read-more">Read More</a>
      <hr>
    </div>
{% endfor %}

This tells Jekyll to run a for-loop before displaying the index.html page. As this loop runs, a div is created for each blog post, along with some info about the post (author, date, etc.). It also truncates the post to 40 words and provides a link to read the full text. A refresh will be pretty blank right now, as there’s not any posts and it looks a bit silly. Let’s create a dummy post to fill things out a bit.

###Create A Post

  • Creating a post is a bit of pretty cool black magic on Jekyll’s side. It expects a specific type of filename and reside in the _posts directory. So let’s create a file named like YEAR-MONTH-DAY-TITLE.md. At the time of this writing, I’m issuing touch 2014-09-07-hello-post-world.md.

  • Inside that file, we’ll add some YAML configuration to the top to tell it some info about layout, title, and author. Edit the file to look like this and feel free to the author and text to whatever you wish:

---
layout: posts
author: Spencer
title: Hello, Post World!
---

Here's some pretty incredible text to prove that it actually shows up in our index page and I'll continue rambling here in hopes that I hit forty words and the post gets truncated. That will allow us to see the "Read More" link appear on our post.
  • Refresh the localhost:4000 page and you’ll see your first post appear!

Nice! Until you click the “Read More” or the title of the post. There’s no posts.html in _layouts, which is what we specified at the top of our file just a second ago. We can’t really use the default layout we created, because it’s missing a couple of things like post title and a back button that we’d want on single posts, but not on the main page.

###Create A Layout For Posts

  • Create a file called posts.html and place it in the _layouts directory. Populate the file with the following:
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
    <meta http-equiv='X-UA-Compatible' content='IE=edge'>
    <meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1.0'>

    <title>{{ site.name }}</title>

    <meta name="author" content="{{ site.name }}" />
    <meta name="description" content="{{ site.description }}">

    <!-- Latest compiled and minified CSS -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">

    <!-- Optional theme -->
    <link rel="stylesheet" href="http://getbootstrap.com/examples/blog/blog.css">
    <link rel="stylesheet" href="/pygments.css">
  </head>

  <body>

  <div class="blog-masthead navbar-fixed-top">
    <div class="container">
      <nav class="blog-nav">
        <a class="blog-nav-item" href="/">Home</a>
        <a class="blog-nav-item" href="#">About</a>
      </nav>
    </div>
  </div>

  <div class="container">
    <div class="blog-header">
      <h1 class="blog-title">{{site.name}}</h1>
      <p class="lead blog-description">{{site.description}}</p>
    </div>

    <div class="row">
      <div class="col-sm-8 blog-main">
          <div class="blog-post">   
            <h2 class="blog-post-title">{{ page.title }}</h2>
            <p class="blog-post-meta">Written {{ page.date | date: "%m.%d.%Y" }} by {{page.author}}</p>
            <hr>
              {{content}}
            <hr>
            <a href="/"><< Back</a>
          </div>
      </div>

      <div class="col-sm-3 col-sm-offset-1 blog-sidebar">
        <div class="sidebar-module sidebar-module-inset">
          <h4>About</h4>
          <p>{{site.about}}</p>
        </div>
        <div class="sidebar-module sidebar-module-inset">
          <h4>Contact Info</h4>
          <ol class="list-unstyled">
            <li><a href="#" target="_blank">Github</a></li>
            <li><a href="#" target="_blank">LinkedIn</a></li>
          </ol>
        </div>
      </div>

    </div>
  </div>

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>


  </body>
</html>
  • Now, let’s click on a link to our post and see it in all of it’s glory!

###Closing Well, that’s pretty much it for how I created my new blog. Now is probably a good time to push all of this to Github and ensure it works properly if you’re following along with these instructions. I hope this has been a helpful write-up and I’m sure I’ll document other cool stuff that I run into as I learn to use Jekyll more effectively. It’s now up to you to create lots and lots of posts on your own blog!

As I wrote previously, I wanted to learn to use Github Pages to build a new blog and start keeping track of my personal projects and ideas. However, in getting started, I noticed that there really wasn’t a straightforward way to move from Github’s quick and dirty tutorial to actually creating a useful blog. There’s some tools like Jekyll Bootstrap, but it seems like it’s been a while since it has been updated. So after combing through a lot of other people’s pages and some Jekyll tutorials, I was finally able to get something presentable up and running. I figured I’d share the overall process I used as a first series of posts, in case someone else finds it useful.

This first post will center around getting a development environment setup and creating a “Hello, World!” type page to demonstrate the basics. So without further ado…

This guide assumes a standard Ubuntu installation and a pre-existing Github account.

###Create Our Github Page

  • First, we’ll want to create an empty repo out on Github. The naming here is quite important, so for a personal blog, you’ll want to name it like GITHUB_USERNAME.github.io. So login to Github and click “+ New Repository”. My repo info looked like this:

  • Now, let’s clone our repo locally. This should be as simple as a git clone https://github.com/USERNAME/USERNAME.github.io.git in some place that you’re comfortable with. If you change into that directory, you’ll see it’s currently empty, unless you created a README or LICENSE file as part of your repo creation.

Note: If you don’t have git installed, issue sudo apt-get install git to install.

###Install The Basics

  • Let’s grab some packages we need to serve our site locally. This will be useful when writing posts, so you don’t have to push to Github constantly just to see what it looks like. Issue sudo apt-get install ruby ruby1.9.1-dev python-pip.

  • We’ll need a Javascript parser and the Jekyll gem, so get NodeJS and Jekyll from rubygems: sudo gem install node jekyll

  • This step is optional for some, but I needed syntax highlighting for code blocks. So install the Pygments package for Python: sudo pip install Pygments

###Create Our Folder Structure

There’s a certain folder structure that Jekyll looks for in your repo. For example, all posts must live in the ‘_posts’ folder and must be titled like YEAR-MONTH-DAY-TITLE.FORMAT. To help understand, the title of this post is 2014-09-06-creating-a-blog-1.md. Anyways, I wanted to get up and running with the absolute basic amount of stuff needed in my repo so here’s what seemed to work.

  • Change into your repo directory in a terminal. Create two directories, _layouts and _posts. They must be named exactly that, including the underscore at the front. Issue mkdir _layouts _posts

  • Let’s also touch a couple of files that we’ll use later on. Issue touch _config.yml index.html

###Configure The Site

  • We could probably do this part much later in the tutorial, but let’s go ahead and put some basic info about our site in the _config.yml file. Feel free to edit these fields to your liking, with the exception of the ‘highlighter’ option. That is for our code highlighting.
name: "Test Blog"

description: "A snazzy new blog, just for me!"

about: "Here's a bit of text that we'll put in an about section later on in our tutorial. I'm intentionally making it a little longer so that the section will look fuller later..."

highlighter: pygments

###Test Our Local Server

  • Now, what’s a tutorial without a “Hello, World!” portion? Let’s go ahead and edit index.html to say that. The “Hello, World!” text should be the only thing in the file.

  • In the root of your repo, let’s tell Jekyll to server our pages so we can see if everything works. Issue jekyll serve --watch. The watch flag will monitor the repo for changed files and regenerate automatically, so you don’t have to start/stop the process. However, I’ve noticed that’s not the case for certain files, namely _config.yml. Your terminal should look something like this when you start the process:

Configuration file: /home/rsmitty/Desktop/rsmitty.github.io/_config.yml
            Source: /home/rsmitty/Desktop/rsmitty.github.io
       Destination: /home/rsmitty/Desktop/rsmitty.github.io/_site
      Generating...
                    done.
 Auto-regeneration: enabled for '/home/rsmitty/Desktop/rsmitty.github.io'
Configuration file: /home/rsmitty/Desktop/rsmitty.github.io/_config.yml
    Server address: http://0.0.0.0:4000/
  Server running... press ctrl-c to stop.
  • Open the address http://localhost:4000 in a browser and you should see your “Hello, World!” text. Neat!

###Check It Out On Github

Okay, so we’ve got our first official page. How do we get that out to Github? Turns out, it’s really easy, because we’ve already got everything laid out just right locally.

  • Add all of our files to be commited: git add --all

  • Commit them with a helpful message: git commit -m "Committing basic folder structure and Hello World page."

  • Push it to Github: git push origin master

  • Give it a few seconds, and you should see your new page at your USERNAME.github.io address:

###Closing That’s it for part one of this tutorial. We’ve now got a development environment setup and know how to push our pages to Github. In part two, we’ll explore how to use Bootstrap to setup a blog layout and create some custom layout pages so that our site looks clean and user friendly.

Welp, after a long time away from anything blog related, I’ve decided to spin up a new site to start trying to keep better documentation of the stuff I do personally. A friend of mine asked about my old blog last week and, after telling me that he had liked to read it, I decided that maybe now was as good a time as any to get back into the swing of things. Life has changed quite a bit since the old entries I had made on my previous site, so the focus of my write-ups here will be pretty drastically different.

I had originally stopped writing mainly because, as graduation neared at Clemson, I was short on time and frantically trying to find a job to transition to ASAP. I also didn’t want to pay yearly for a domain name and to host my site somewhere. So in getting back into this, I have made the transition to Github Pages as a free spot to host my stuff. Pages uses the Jekyll framework to host static sites, which sounded like a neat platform to try out. This has also given me a chance to use git in a different way and learn the markdown syntax. I hope that I’ll be able to expand on these skills more in the future.

As for most of my new posts, they’ll probably center around the creation of this site (it took me a while to figure out the simplest path forward for Jekyll + Bootstrap), Linux system administration, OpenStack oddities and how-tos, and automation tools like Chef and Ansible. Finally, in regards to my older stuff, I was able to dig the old site out of the archive.org Wayback Machine (hooray for not keeping backups), so I’ve ported a few of the ones that were still somewhat interesting to this site.

Hope you like it!

Back in CPSC 101, we did a lot of image editing. We only used PPM images in class and I was thinking the other day about how even though I hated the project at the time, it was pretty interesting. So, I decided to re-code some of it in Python. Basically, all I’ve done is written a script that will allow the conversion of any color PPM image into greyscale. I feel like this can be pretty handy, as it’s fast and seems to do a pretty good job.

Here’s the before image (Ironically, I had to convert to another image format before posting):

alt text

And here’s the output:

alt text

A few particulars about the files and scripts:

  • I used the P3 header, which is easier because it allows you just to write ASCII into the file.

  • Also, you must ensure that the header of the input photo must only have 3 lines. I’ve found that if you convert a JPEG from Gimp, it adds its on line in the header. It was easy to just do ‘nano in.ppm’ in terminal and remove the line they put in.

  • The script can be run using ‘python grey.py <input filename>’

  • This ONLY works on PPM images. Maybe someday I’ll look into another image format.

And that’s it!

Here’s the code, check it out:

#Greyscale Converter
#Spencer Smith
#Mar. 22, 2010
import sys
 
#Open file
FILE = open(sys.argv[1],"r")
 
#Read input file
indata = FILE.readlines()
 
#Strip header
del indata[0]
size = indata[0]
del indata[0:2]
 
#Open output image
GREY = open("out_grey.ppm","w")
 
#Write header to output image
GREY.write("P3\n")
GREY.write(size)
GREY.write("255\n")
 
#Calculate and write RGB values
counter = 0
for x in indata:
  val = x
  if counter % 3 == 0:
    val1 = int(val)*.3
  elif counter % 3 == 1:
    val2 = int(val)*.59
  elif counter % 3 == 2:
    val3 = int(val)*.11
  px = val1+val2+val3
  GREY.write("%d\n%d\n%d\n" % (int(px),int(px),int(px)))
  counter = counter + 1
 
#Close files
FILE.close()
GREY.close()