Using CVSNTs History SQL tables instead of parsing the CVS Logs

NOT 3.7 compliant!!!!!
Working on it though! 

CVSNT will automatically create SQL records for the cvs history.
We have one very large project that when attempting to access the cvs repository through Jira, never finished parsing the logs.
I changed the Jira code to allow us to take advantage of these logs thus allowing that large project to use the CVS repository functionality of Jira and eliminating the 1 hour (or whatever your setting is) delay between CVS commits and the version control information showing up in Jira.

I did this the way that I did it in a fairly hacked way because I was told that the version control for Jira was to be rewritten in the future as all plugins, so I did not want to change too much code or spend too much time on something that will be radically changed later on.

Obviously you need a datasource for the cvs history log sql database

Essentially I added the ability to have a hardcoded value of CVSNTAUDIT for the log file path in the cvs repository admin screen
if this is set, then it will ignore parsing and when asked to retrieve the version control info it will get it from the database.

Here is that story:

1: /jira/src/test/test/local/jira/web/action/vcs/TestSelectProjectRepository.java
(We need the actual issue, not just the issue key)
change the line from

public List getCommitsForIssue(String issueKey)

to

public List getCommitsForIssue(GenericValue issue)

2: /jira/src/test/test/local/jira/web/action/admin/vcs/TestRepositoryActionSupport.java
(same reason as above)
change the line from

public List getCommitsForIssue(String issueKey) throws RepositoryException

to

public List getCommitsForIssue(GenericValue issue) throws RepositoryException

3: /jira/src/java/com/atlassian/jira/vcs/Repository.java
(same reason as above)
add an import for GenericValue and
change the line from

public List getCommitsForIssue(String issueKey) throws RepositoryException;

to

public List getCommitsForIssue(GenericValue issue) throws RepositoryException;

4: /jira/src/java/com/atlassian/jira/issue/managers/DefaultActionManager.java
(same reason as above)
change the line from

List coms = repository.getCommitsForIssue(issue.getString("key"));

to

List coms = repository.getCommitsForIssue(issue);

5: /jira/src/java/com/atlassian/jira/web/action/admin/vcs/RepositoryTest.java
(allow the option of using the sql tables instead of the log files)
make the following change

            if (RepositoryManager.CVS_TYPE.equals(repository.getType()))
            {
                CvsRepository cvsRepository = (CvsRepository) repository;
                try
                {
...
            }
            else
            {
                message = "Unknown repository type.";
            }

to

            if (RepositoryManager.CVS_TYPE.equals(repository.getType()))
            {
            	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
                CvsRepository cvsRepository = (CvsRepository) repository;
            	if(cvsRepository.getCvsLogFilePath().indexOf("CVSNTAUDIT") == -1) {
            	try
                {
...
            }
            else
            {
                message = "Unknown repository type.";
            }

6: /jira/src/java/com/atlassian/jira/web/action/admin/vcs/RepositoryActionSupport.java
make the following changes

protected void testRepository(String repositoryName, String logFilePath, String cvsRoot, String moduleName, String password, int cvsTimeout, boolean fetchLog) throws AuthenticationException, IOException, CommandException, LogSyntaxException, LockException
    {
        final FileImpl logFile = new FileImpl(logFilePath);
        // Only check if we can get the log if the user wants us to do it
        if (fetchLog)
        {
...
    }


private void checkLogFile()
    {
        try
        {
...
    }

to

import com.atlassian.jira.config.properties.APKeys;


protected void testRepository(String repositoryName, String logFilePath, String cvsRoot, String moduleName, String password, int cvsTimeout, boolean fetchLog) throws AuthenticationException, IOException, CommandException, LogSyntaxException, LockException
    {
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(logFilePath.indexOf("CVSNTAUDIT") == -1) {
    	final FileImpl logFile = new FileImpl(logFilePath);
        // Only check if we can get the log if the user wants us to do it
        if (fetchLog)
...
    	}
    }

private void checkLogFile()
    {
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(getLogFilePath().indexOf("CVSNTAUDIT") == -1 && getLogFilePath().indexOf("SVNREPOSITORY") == -1) {
        try
        {
...
    	}
    }

7: /jira/src/java/com/atlassian/jira/vcs/cvsimpl/CvsRepository.java
make the following changes

    private void parseCvsLogs(String filename) throws IOException, LogSyntaxException, LockException
    {
        this.content = cvsRepositoryUtil.parseCvsLogs(new FileImpl(filename), moduleName, cvsRepositoryUtil.parseCvsRoot(cvsRoot).getRepository(), getName());
    }

    /**
     * The filename where the log has been written
     */
    public synchronized String updateCvs() throws CommandException, AuthenticationException, IOException, LockException
    {
        if (fetchLog)
...
    }

    /**
     * Return a list of {@link Commit} objects
     *
     * @param issueKey
     */
    public List getCommitsForIssue(final String issueKey) throws RepositoryException
    {
        log.debug("Starting commit matching.");
        long t0 = System.currentTimeMillis();
        // We should always let the VcsService update the cvs logs, doing it synchronously here can lock
        // up the UI, JRA-8857
        if (content == null)
...
        return cvsCommits;
    }

    public synchronized void updateRepository() throws CommandException, AuthenticationException, IOException, LockException, LogSyntaxException
    {
        final String cvsLog = updateCvs();
        parseCvsLogs(cvsLog);
    }

to

   import org.ofbiz.core.entity.GenericValue;
   import weblogic.jndi.Environment;
   import java.sql.Connection;
   import java.sql.ResultSet;
   import java.sql.SQLException;
   import java.sql.Statement;
   import java.sql.Timestamp;
   import javax.naming.Context;
   import javax.naming.NamingException;
   import javax.sql.DataSource;

    private void parseCvsLogs(String filename) throws IOException, LogSyntaxException, LockException
    {
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(filename.indexOf("CVSNTAUDIT") == -1) {
    		this.content = cvsRepositoryUtil.parseCvsLogs(new FileImpl(filename), moduleName, cvsRepositoryUtil.parseCvsRoot(cvsRoot).getRepository(), getName());
    	}
    }

    /**
     * The filename where the log has been written
     */
    public synchronized String updateCvs() throws CommandException, AuthenticationException, IOException, LockException
    {
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(cvsLogFilePath.indexOf("CVSNTAUDIT") == -1) {
        if (fetchLog)
...
    	} else {
    		log.debug("Not fetching log as this repository is uding the CVSNT Audit Database");
    		return cvsLogFilePath;
    	}
    }

    /**
     * Return a list of {@link Commit} objects
     *
     * @param issueKey
     */
    public List getCommitsForIssue(final GenericValue issue) throws RepositoryException
    {
        log.debug("Starting commit matching.");
        long t0 = System.currentTimeMillis();
        // We should always let the VcsService update the cvs logs, doing it synchronously here can lock
        // up the UI, JRA-8857
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(cvsLogFilePath.indexOf("CVSNTAUDIT") == -1) {


        if (content == null)
...
        return cvsCommits;
    	} else {
          List matchingCommits = createCommitList(issue);

          if (log.isDebugEnabled())
          {
              log.debug("Finished commit matching.");
              log.debug("Matching took " + (System.currentTimeMillis() - t0) + "ms and matched " + matchingCommits.size() + " commits.");
          }
          return matchingCommits;
    	}
    }

    public synchronized void updateRepository() throws CommandException, AuthenticationException, IOException, LockException, LogSyntaxException
    {
    	//PUT THE STRING CVSNTAUDIT in the filename if you dont want it to parse the cvs logs, but rather use the CVSNT audit database
    	if(cvsLogFilePath.indexOf("CVSNTAUDIT") == -1) {
        final String cvsLog = updateCvs();
        parseCvsLogs(cvsLog);
    	}
    }

    public List createCommitList(GenericValue issue) {
    	List result = new ArrayList();
		Connection databaseConnection = null;
		String shopFindQuery = "";
		Timestamp ts = issue.getTimestamp("created");
		Timestamp ts2 = issue.getTimestamp("updated");
		//subtract one day from the create day and add one day to the update day
		//to be sure we get the changes that may be off because of server time
		//differences or developer habits
		ts.setTime(ts.getTime() - 84600000);
		ts2.setTime(ts2.getTime() + 84600000);



		shopFindQuery = "select * "
			 + " from cvsnt.CommitLog hl"
			 + " join cvsnt.SessionLog sl"
			 + " on sl.id = hl.sessionid"
			 + " where sl.Date >= '" + ts.toString() + "'"
			 + " AND sl.VirtRepos = '/" + name + "'"
			 + " AND hl.message like '%" + issue.getString("key") + "%'"
		     + " AND hl.message not like '%" + issue.getString("key") + "[0-9]%'";
			shopFindQuery+=" and sl.Date <= '" + ts2.toString() + "'";
			shopFindQuery+=" AND sl.command = 'commit'";



		Statement sStmt = null;
		ResultSet rs = null;
		try {

			Environment environment = new Environment();
			environment.setProviderUrl(System.getProperty("smlocalserverurl"));
			environment.setSecurityPrincipal(null);
			environment.setSecurityCredentials(null);

			Context context = environment.getInitialContext();

			databaseConnection = ((DataSource) context.lookup("CVSDS")).getConnection();
			databaseConnection.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
			databaseConnection.setAutoCommit(true);

			sStmt = databaseConnection.createStatement();
			rs = sStmt.executeQuery(shopFindQuery);
			while (rs.next()) {
				Collection revisions = new ArrayList();
				com.atlassian.jira.vcs.cvsimpl.CvsRevision revision = new com.atlassian.jira.vcs.cvsimpl.CvsRevision();
				revision.setRevision(rs.getString("NewRev"));
				revision.setLinesAdded(rs.getInt("Added"));
				revision.setLinesRemoved(rs.getInt("Removed"));
				CvsFile cvsfile = new CvsFile();
				cvsfile.setFilenameWithPath(rs.getString("Directory") + "/" + rs.getString("FileName"));
				revision.setFile(cvsfile);
				revisions.add(revision);
				Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
				CVSNTCommit commit = new CVSNTCommit(rs.getDate("Date", cal), rs.getString("UserName"), rs.getString("message"), rs.getString("tag"), revisions);
				result.add(commit);
			}
		} catch (SQLException sqle) {
			sqle.printStackTrace();
		} catch (NamingException ne) {
			ne.printStackTrace();
		} finally {
			try {
				if (rs != null)
					rs.close();
				if (sStmt != null)
					sStmt.close();
				if (databaseConnection != null)
					databaseConnection.close();
			} catch (SQLException e) {
				e.printStackTrace();
			}
		}
    	return result;
    }

8: new file /jira/src/java/com/atlassian/jira/vcs/cvsimpl/CvsRevision.java

package com.atlassian.jira.vcs.cvsimpl;

public class CvsRevision {
	public CvsFile file;
	public String revision;
	public int linesAdded;
	public int linesRemoved;
	public CvsFile getFile() {
		return file;
	}
	public void setFile(CvsFile file) {
		this.file = file;
	}
	public int getLinesAdded() {
		return linesAdded;
	}
	public void setLinesAdded(int linesAdded) {
		this.linesAdded = linesAdded;
	}
	public int getLinesRemoved() {
		return linesRemoved;
	}
	public void setLinesRemoved(int linesRemoved) {
		this.linesRemoved = linesRemoved;
	}
	public String getRevision() {
		return revision;
	}
	public void setRevision(String revision) {
		this.revision = revision;
	}

}

9: new file /jira/src/java/com/atlassian/jira/vcs/cvsimpl/CvsFile.java

package com.atlassian.jira.vcs.cvsimpl;

public class CvsFile {
	public String filenameWithPath;

	public String getFilenameWithPath() {
		return filenameWithPath;
	}

	public void setFilenameWithPath(String filenameWithPath) {
		this.filenameWithPath = filenameWithPath;
	}
}

10: new file /jira/src/java/com/atlassian/jira/vcs/cvsimpl/CVSNTCommit.java

package com.atlassian.jira.vcs.cvsimpl;

import net.sf.statcvs.model.Commit;

import com.atlassian.jira.vcs.VCSCommit;

import java.util.*;

public class CVSNTCommit implements VCSCommit
{

	private Date date;
	private String author;
	private String comment;
	private String branchName;
	private Collection revisions;
	public CVSNTCommit(Date date, String author, String comment, String branchName, Collection revisions)
    {
        this.date = date;
        this.author = author;
        this.comment = comment;
        this.branchName = branchName;
        this.revisions = revisions;
    }

    public Date getDate()
    {
        return date;
    }

    public String getAuthor()
    {
        return author;
    }

    public Collection getRevisions()
    {
        return revisions;
    }

    public String getComment()
    {
        return comment;
    }

    public String getBranchName()
    {
        return branchName;
    }
}
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Jan 03, 2007

    Chris Houghten says:

    Ugh... This soooo does not work with 3.7 because of refactoring Atlassian did wi...

    Ugh... This soooo does not work with 3.7 because of refactoring Atlassian did with thier code.
    I just merged the 3.7 code with all of my stuff and found this out.
    I am attempting to get it working with 3.7 and will post here when it does.