# Data Load error - Job ID Hashing

**URL:** https://forum.squirro.com/t/data-load-error-job-id-hashing/92
**Category:** The Insight Engine
**Created:** [March 1, 2022, 7:08pm UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92 "2022-03-01T19:08:12Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![manutej](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.squirro.com/manutej/32/59_2.png) [@manutej](https://forum.squirro.com/u/manutej)
#### Post date: [March 1, 2022, 7:08pm UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92/1 "2022-03-01T19:08:12Z")

</div>

Hello,  
I am getting an error for my data loader python file based on Job ID: It says “TypeError: Unicode-objects must be encoded before hashing”, does anyone have experience with this error? I am using standard template code for this function:

```auto
    def getJobId(self):
        """
        Return a unique string for each different select
        :returns a string
        """
        # Generate a stable id that changes with the main parameters
        # m = hashlib.sha256()
        m = hashlib.blake2b(digest_size=20)
        m.update(repr(os.getcwd()))
        job_id = m.hexdigest()
        log.debug("Job ID: %s", job_id)
        return job_id

```

ERROR:

```auto
Traceback (most recent call last):
  File "/Users/manutej.mulaveesala/python_virtual_environments/squirro/lib/python3.7/site-packages/squirro/dataloader/sq_data_load.py", line 693, in main
    launcher.execute()
  File "/Users/manutej.mulaveesala/python_virtual_environments/squirro/lib/python3.7/site-packages/squirro/dataloader/sq_data_load.py", line 472, in execute
    self.config.source, cli_mode=cli_mode, max_inc_value=max_inc_value
  File "/Users/manutej.mulaveesala/python_virtual_environments/squirro/lib/python3.7/site-packages/squirro/dataloader/sq_data_load.py", line 148, in load_from_source
    job_id = self._get_job_id(source, source_name)
  File "/Users/manutej.mulaveesala/python_virtual_environments/squirro/lib/python3.7/site-packages/squirro/dataloader/sq_data_load.py", line 337, in _get_job_id
    job_id = source.getJobId()
  File "nytimes_dataloader.py", line 179, in getJobId
    m.update(repr(os.getcwd()))
TypeError: Unicode-objects must be encoded before hashing

```

---

<div class="post-metadata">

### Author: ![manutej](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.squirro.com/manutej/32/59_2.png) [@manutej](https://forum.squirro.com/u/manutej)
#### Post date: [March 2, 2022, 1:58am UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92/2 "2022-03-02T01:58:52Z")

</div>

I was able to figure out the issue. The reason was some of the code was outdated python2 code and needed to be adjusted. The key change being that the query needs to be encoded as utf-8 explicitly rather than implicitly as was previously possible with python2.

See below:

```auto
    def getJobId(self):
        """
        Return a unique string for each different select
        :returns a string
        """
        # Generate a stable id that changes with the main parameters

        m = hashlib.sha256()
        m.update(self.args.query[0].encode("utf-8"))
        job_id = m.hexdigest()
        log.debug("Job ID: %s", job_id)
        return job_id

```

---

<div class="post-metadata">

### Author: ![pneff](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.squirro.com/pneff/32/11_2.png) [@pneff](https://forum.squirro.com/u/pneff)
#### Post date: [March 2, 2022, 8:51am UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92/4 "2022-03-02T08:51:03Z")

</div>

Where did you get this template code from?

The template I’m aware of at [training/custom\_plugin.py at 0422e9eaa291c63063712d286ff5addd965b8546 · squirro/training · GitHub](https://github.com/squirro/training/blob/0422e9eaa291c63063712d286ff5addd965b8546/templates/data/custom_dataloader_plugin/custom_plugin.py#L80-L94) has the right handling, which is to use `repr` and `encode` in combination to get the right input to the hashing function.

Your revised code does not necessarily handle the arguments correctly. It seems that `args.query` is a multi-value option, but only the first value will be used for the job id.

---

<div class="post-metadata">

### Author: ![filippo82](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.squirro.com/filippo82/32/53_2.png) [@filippo82](https://forum.squirro.com/u/filippo82)
#### Post date: [March 3, 2022, 9:16am UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92/5 "2022-03-03T09:16:37Z")

</div>

You can also find additional information on this page: [https://squirro.atlassian.net/wiki/spaces/DOC/pages/2425979318/Writing+a+custom+one-click+connector#getJobId](https://squirro.atlassian.net/wiki/spaces/DOC/pages/2425979318/Writing+a+custom+one-click+connector#getJobId)

---

<div class="post-metadata">

### Author: ![manutej](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.squirro.com/manutej/32/59_2.png) [@manutej](https://forum.squirro.com/u/manutej)
#### Post date: [March 4, 2022, 1:46pm UTC](https://forum.squirro.com/t/data-load-error-job-id-hashing/92/6 "2022-03-04T13:46:27Z")

</div>

Yes I saw that Patrice. Did not realize that was the standard code that should always be in there.  
What does Blake2b do?

I used another version because I didn’t realize this piece shouldn’t be customized. Good to know thanks

The first custom param and second custom param Is where I would include my arguments?
